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Abstract 

This paper investigates the enumeration, rate region computation, and hierarchy of general multi-source multi-sink 
hyperedge networks under network coding, which includes multiple network models, such as independent distributed 
storage systems and index coding problems, as special cases. A notion of minimal networks and a notion of network 
equivalence under group action are defined. An efficient algorithm capable of directly listing single minimal canonical 
representatives from each network equivalence class is presented and utilized to list all minimal canonical networks 
with up to 5 sources and hyperedges. Computational tools are then applied to obtain the rate regions of all of these 
canonical networks, providing exact expressions for 744,119 newly solved network coding rate regions corresponding 
to more than 2 trillion isomorphic network coding problems. In order to better understand and analyze the huge 
repository of rate regions through hierarchy, several embedding and combination operations are defined so that the 
rate region of the network after operation can be derived from the rate regions of networks involved in the operation. 
The embedding operations enable the definition and determination of a list of forbidden network minors for the 
sufficiency of classes of linear codes. The combination operations enable the rate regions of some larger networks 
to be obtained as the combination of the rate regions of smaller networks. The integration of both the combinations 
and embedding operators is then shown to enable the calculation of rate regions for many networks not reachable 
via combination operations alone. 


I. Introduction 

Many important practical problems, including efficient information transfer over networks 0 0 the design 
of efficient distributed information storage systems and the design of streaming media systems ®-(Tg, 

have been shown to involve determining the rate region of an abstracted network under network coding. Yan et 
aEs celebrated paper GD has provided an exact representation of these rate regions of networks under network 
coding. Their essential result is that the rate region of a network can be expressed as the intersection of the region 
of entropic vectors |T3| with a series of linear (in)equality constraints created by the network’s topology and 
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the sink-source requirements, followed by a projection of the result onto the entropies of the sources and edge 
variables. However, this is only an implicit description of the rate region, because the region of entropic vectors 
is still unknown for > 4. 

Nevertheless, as we have previously demonstrated in (T4) -fT^, through the use of appropriate inner and outer 

this implicit formulation can be used to develop algorithms by which a 


bounds to that we will review in §II-C 


computer can very rapidly calculate the rate region, its proof, and the class of capacity achieving codes, for small 
networks, each of which would previously have taken a trained information theorist hours or longer to derive. While 
the development of this rate region calculation, code selection, and converse proof generation algorithm eg is not 
the focus of the present paper, it involves developing techniques to derive and project polyhedral inner and outer 
bound descriptions for constrained regions of entropic vectors. When it comes to rate regions, ^and |II-C| of the 
paper will focus more on exactly what was calculated, what can be calculated, and, later in the paper, what can be 
learned from the resulting rate regions, rather than the exact computations by which the rate regions were reached. 
The rate region algorithm design and specialization, which involve a separate and parallel line of investigation, are 
left to discussion by another series of papers 

The ability to calculate network coding rate regions for small networks rapidly with a computer motivates an 
alternative, more computationally thinking oriented, agenda to the study of network coding rate regions. At the 
beginning, one’s goal is to demonstrate the method’s power by applying the algorithm to derive the rate region of 
as many networks and applications of network coding as possible. To do this, in ^ we first slightly generalize 
Yeung’s labeled directed acyclic graph (DAG) model for a network coding problem ( (H), Ch. 21) to a directed 
acyclic hypergraph context, then demonstrate how the enlarged model handles as special cases the wide variety 
of other models in applications in which network coding is being employed, including, but not limited to, index 
coding, multilevel diversity coding systems (MDCS), and distributed storage. 

With the slightly more general model in hand, the first issue in the computationally thinking oriented agenda is 
network generation and enumeration, i.e., how to list all of the networks falling in this model class. In order to avoid 
repetitive work, thereby reaching the largest number of networks possible with a constant amount of computation, 
it is desirable to understand precisely when two instances of this model (i.e., two network coding problems) are 
equivalent to one another, in the sense that the solution to one directly provides a solution to another. This notion of 
network coding problem equivalence, which provides a very different approach but is in the same high level spirit as 
the transformation of network channel coding problems to network coding problems in | [2Q| and the transformation 
of network coding problems to index coding problems in (H), will be revisited at multiple points of the paper, 
beginning in this present context of enumeration, but also playing an important role in the discussion of hierarchy 
later. 

The first notion of equivalence we develop is that of minimality, by removing any redundant or unnecessary 
parts in the network instance. Partially owing to the generality of the network coding problem model, many valid 
instances of it include within a network parts which can be immediately detected as extraneous to the determination 
of the instance’s rate region. In this sense, an instance is directly reducible to another smaller instance by removing 
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completely unnecessary and unhelpful sources, nodes, or edges. In order to provide the smallest possible instance by 
not including these extraneous components, we formalize in ^nl|the notion of network minimality, listing a series of 
conditions which a network coding problem description must obey to not contain any obviously extraneous sources, 
nodes, or edges. 

The next notion of equivalence looks to symmetry or isomorphism between problem descriptions. Beginning by 
observing that a network must be labeled to specify its graph, source availability, and demands to the computer, yet 


the underlying network coding problem is insensitive to the selection of these labels, we define in §IV-B a notion 
of network coding problem equivalence through the isomorphism associated with the selection of these labels. We 
review that the proper way to formalize this notion is through identifying equivalence classes with orbits under group 
actions. A naive algorithm to provide the list of network coding problems to the rate region computation software 
would simply list all possible labeled network coding problems, test for isomorphism, then narrow the list down 
to only those which are inequivalent to one another, keeping only one element, the canonical representative, of the 
equivalence class. However, the key reason for formalizing this notion of equivalence is that the number of labeled 
network coding problem instances explodes far faster than the number of network coding problem equivalence 
classes. Hence, we develop a better technique for generating lists of canonical network coding problem instances 
by harnessing techniques and algorithms from computational group theory that enable us to directly list the minimal 
canonical representatives of the network coding problem equivalence classes as described in |IV-C[ 

With the list of all minimal canonical network coding problems up to a certain size in hand, we can utilize our 
algorithm and software to calculate the rate region bounds, the Pareto optimal network codes, and the converse 
proofs, for each, building a very large database of rate regions of network coding problems up to this size. Owing 
to the variety of the model, even for tiny problems, this database quickly grows very large relative to what a 
human would want to read through. For instance, our previous paper applying this computational agenda to the 
narrower class of MDCS problems |T^ , yielded the rate regions of 6,868 equivalence classes of MDCS problems 
and bounds for 492 more MDCS problem equivalence classes, while the database developed in this paper contains 
the rate regions of 744,119 equivalence classes of network coding problems. These equivalence classes of networks 
correspond to solutions for 9,050,490 network coding problems with graphs specified via edge dependences and 
2,381,624,632,119 network coding problems specified in the typical node representation of a graph. While it is 
possible to use the database to report statistics regarding the sufficiency of certain classes of codes as will be done 
in ^Vj in order to more meaningfully enable humans to learn from the database, as well as from the computational 
research, one must utilize some notion of network structure to organize it for analysis. 

Our method of endowing structure on the set of network coding problems is through hierarchy, in which we 
explain the properties and/or rate regions of larger networks as being inherited from smaller networks (or vice- 
versa). Of course, part of a network coding problem is the network graph, and further, network coding and entropy 
is related to matroids, and these nearby fields of graph theory and matroid theory have both undergone a thorough 
study of hierarchy which directly inspires our approach to it. In graph theory, this notion of hierarchy is achieved by 
recognizing smaller graphs within large graphs which can be created by deleting or contracting the larger graph’s 





4 


edges, called minors, and is directly associated with a crowning achievement. Namely, the celebrated well-quasi- 
ordering result of graph theory | [^ , showed that any minor closed family of graphs (i.e., ones for which any 
minor of a graph in that family is also in the family) has at most a finite list of forbidden minors, which no graphs 
in that family can contain. While the families of minor closed graphs are typically infinite, they are then, in a sense, 
capable of being studied through a finite object which is their forbidden minors: if a graph does not have one of 
these forbidden minors, it is then in the family. In matroid theory, which in a certain sense extends graph theory, 
one has a similar notion of hierarchy endowed through matroid minors, generated through matroid contraction and 
deletion. While one can generate minor closed families of matroids with an infinite series of forbidden minors, the 
celebrated, and possibly recently proved. Rota’s conjecture p4| , | [25| , stated that those matroids capable of being 
represented over a particular finite field have at most a finite list of forbidden minors. In this paper, inspired by 
these hierarchy theories in graphs and matroids, we aim to derive a notion of network minors created from a series 
of contraction and deletion-like operators that shrink network coding problems, called embedding operators, as well 
as operations for building larger network coding problems from smaller ones, called combination operators. These 
operators work together to build our notion of minors and a sense of hierarchy among network coding problems. 

Developing a notion of network coding problem hierarchy is important for several reasons. First of all, as explained 
above, even after one has calculated the rate regions of all networks up to a certain size, it is of interest to make 
sense of this very large quantity of information by studying its structure, and hierarchy is one way of creating a 
notion of structure. Second of all, the computational techniques for proving network rate regions can only handle 
networks with tens of “variables”, the sum of the number of sources and number of hyperedges in the graph, and 
hence are limited to direct computation of fairly small problem instances. If one wants to be able to utilize the 
information gathered about these small networks to understand the rate regions of networks at scale, one needs 
methods for putting the smaller networks together into larger networks in a way such that the rate region of the 
larger network can be directly calculated from those of the smaller networks. 

Our embedding operators, defined and discussed in extend the series of embedding operations we had for 
MDCS problems in |T^ , and augment them, to provide methods for obtaining small networks from big networks 
in such a way that the rate region of the smaller network, and its properties, are directly inherited from the larger 
network. Our combinations operators, discussed in |VII| work in the opposite direction: they provide methods for 
putting together smaller networks to make larger networks in such a way that the rate region of the larger network 
can be directly calculated from the rate region of the smaller networks. Both of our lists of operators are small and 
somewhat simple, however, when they work together, they provide a very powerful way of endowing hierarchical 
structure in network coding problems. In particular, the joint use of the combination and embedding operators 
provide a very powerful way of obtaining rate regions of large networks from small ones, as well as describing 
the properties of families of network coding problems, as we demonstrate in |VIII| They open the door to many 
new avenues of network coding research, and we shall describe briefiy some of the related future problems for 


investigation in ^IX 
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Fig. 1: A general network model A. 

11. Background: Network Coding Problem Model, Capacity Region, and Bounds 

The class of problems under study in this paper are the rate regions of multi-source multi-sink network coding 
problems with hyperedges, which we hereafter refer to as the hyperedge MSNC problems. For ease of reading the 
paper, a notation table is presented in Table [T| A network coding problem in this class, denoted by the symbol 
A, includes a directed acyclic hypergraph {Y^ S’) | [^ as in Fig.[^ consisting of a set of nodes Y and a set S of 
directed hyperedges in the form of ordered pairs e = (u, with v and ^ y\v. The nodes y in the graph 
are partitioned into the set of source nodes intermediate nodes and sink nodes i.e., y = ^ U ^ U 
Each of the source nodes s e ^ will have a single outgoing edge (s, G S. The source nodes in ^ have no 
incoming edges, the sink nodes have no outgoing edges, and the intermediate nodes ^ have both incoming 
and outgoing edges. The number of sources will be denoted by |^| = K, and each source node s e ^ will be 
associated with an independent random variable Ys, s e with entropy H{Ys), and an associated independent 
and identically distributed (IID) temporal sequence of random values. For every source s G define Out(s) to be 
its single outgoing edge, which is connected to a subset of intermediate nodes and sink nodes. A hyperedge e e S 
connects a source, or an intermediate node to a subset of non-source nodes, i.e., e = (i, ^), where i e and 

^ C U \ i). For brevity, we will refer to hyperedges as edges if there is no confusion. For an intermediate 
node g e W, we denote its incoming edges as In(^) and outgoing edges as Ou.t{g). For each edge e = 
the associated random variable Ue = /e(In(i)) is a function of all the inputs of node i, obeying the edge capacity 
constraint Re > H{Ue). The tail (head) node of edge e is denoted as Tl(e) (Hd(e)). For notational simplicity, the 
unique outgoing edge of each source node will be the source random variable, Ue = Yg if Tl(e) = s, denoting 
Ss = {e e S\Tl{e) = s, 5 G to be the variables associated with outgoing edges of sources, and Su = S\Ss io 
be the non-source edge random variables. For each sink t G the collection of sources this sink will demand will 
be labeled by the non-empty set /3{t) C Thus, a network can be represented as a tuple A = 5^, S^^), 

where /3 = t G S^). Note that, though this commonly used node-representation of a network is convenient for 
understanding the network topology, we will use a more concrete representation in ^TV] for enumeration of network 
instances. For convenience, networks with K sources and L = \Su\ edges are referred as (K^L) instances. 
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TABLE I: Notation table 


A 


[xa\a G s^] 

/3 

d, d, /, TT 

9,'^ 

G,S, (c/i,-- - ,3it) 
if(-),h, V 

Hd(e),Tl(e) 

In(g),Out(g) 

y 

K,L,N 

M,^ 

M 

minimal(A'), minimal^/ ^a(^*(A')) 




p,P 


ru 

r* ,f* 


Ar,A^'’ A^,oo’ N 


Re: Ft 

T-,w,Proj^_^(-) 

^c(A),^*(A),^o(A),^«,<„^^',^g, ^linear 
S,J?^ 

t, AT 
T 

r 

y, V 

x,y 

X,Y 




a network instance 
general sets 

vector with elements Xa indexed by/ for each a G ^ 
demands of sink nodes (jnj 
mappings or functions 

an (hyper)edge or encoder, set of all (hyper)edges or encoders (^II| 
head nodes of a hyperedge (§11 


finite field of order q ( ^II-B 
an intermediate node, set of intermediate nodes 
acting group, symmetric group, group generated by ,... , (§IV-A| 


entropy function, an entropy vector, coordinate in h associated with (§II| 
head nodes, tail node of edge e (^n| 
incoming, outgoing edges of node g 
general index terms 


independent set (5II-C 


number of sources, intermediate (hyper)edges/encoders, and total variables in a network 
(A/ = a: + L) (0 


sets associated with network constraints II-B 


a matroid, ground set of a matroid (§ II-C 


increased dimension of space in calculating rate region ({ II-B 


function to reduce a network A' to its minimal representation and associated rate region 
operators (pi) 


collection of variables in a network II-B 


collection of networks in an equivalence class under our definition (^ IV-B 


error probability, probability function ({ II-B i 


all size i subsets of a given set ^ (^ IV-C > 


edge encodings for a network topology and sink demands (5IV-A 


rank function of a matroid M (UI-C 


region of entropic vectors on N variables, its closure. Shannon outer bound, in¬ 
ner bound from Fg-representable matroids on N elements, inner bound from Fg- 
representable matroids on N' elements,inner bound from Fg-representable matroids 
on infinity number of elements, inner bound from linear subspace arrangement ( yi-C| 


edge capacity on edge e, rate vector (UI-B 


dimensions associated with all edge capacities and all source entropies, projection 
operator with the projecting dimensions are associated with r, u) (§II-B| 


rate region or bounds associated with Fiv, 


(5 II-B 


a source node, set of all sources (§II| 
a sink node, set of all sinks (^ 
canonical representatives, i.e., transversal, output from Leiterspiel algorithm (§IV-C 


edge variable, support set of U (^ II-B 
set of all nodes in a network ( 
a vector space, multiple vector spaces 
random variables (' 


jn_ 


vectors of variables (5 II-B 


general set, support set on Y ({ II-B 


collection of network instances ({ IV-C 
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As our focus in the manuscript will be on rate regions of a very similar form to those in pT| , |T3| , this network 
coding problem model is as close to the original one in |TT| , | p3| as possible while covering the multiple instances of 
applications in network coding in which the same message can be overheard by multiple parties. These applications 
include index coding, wireless network coding, independent distributed source coding, and distributed storage. The 
simplest and most direct model change to incorporate this capability is to switch from directed acyclic graphs to 
directed acyclic hypergraphs. As we shall see in |II-B| this small change to the model is easily reconciled with the 
network coding rate region expression and its proof from GD (H) 

A. Special Network Classes 

The network coding problem model just described has been selected to be general enough to include a variety 
of models from the applications of network coding as special cases. A few of these special cases that will be of 
interest in examples later in the paper are reviewed here, including a description of the extra restrictions on the 
model to fall into this subclass of problems. 

Example 1 (Yan Yeung Zhang MSNC): The network model in where the edges are not hyperedges and 
the outputs of a source node can be multiple functions of the source, can be viewed as a special class of 
networks in our model. This is because the sources can be viewed as intermediate nodes and a virtual 
source node connecting to each of them can be added. For instance, a small network instance of the model 
in Q^, as shown in Fig.|^ can be viewed as a hyperedge network instance introduced in this paper, by 
adding virtual source nodes si',s2' to sources si,s2, respectively. 

Example 2 (Independent Distributed Source Coding): The independent distributed source coding (IDSC) 
problems, which were motivated from satellite communication systems (^, can be viewed as a special 
class of networks In our model. They are three-layer networks, where sources are connected with some 
intermediate nodes and those intermediate nodes will transmit coded messages to sinks. For instance, the 
IDSC problem in Fig.j^can be converted to a hyperedge multi-source network coding problem. As a special 
class of IDSC problems with decoding priorities among sources, the multi-level diversity coding systems 
(T^ , ( 2 ^ are naturally a class of networks in our current general model. In the experimental results section 
^ we will not only show results on general hyperedge networks, but also some results on IDSC problems. 









Vi 
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Fig. 3: An IDSC problem can be viewed as a hyperedge MSNC. 


Yi Y2 I3 



Fig. 4: An index coding problem is a three-layer hyperedge MSNC with only one intermediate edge. 


Example 3 (Index Coding): Since direct access to sources as side information is allowed in our network 
model, index coding problems are also a special class of our model with only one intermediate edge. That 
is, a iT-source index coding problem can be viewed as a {K, 1) hyperedge MSNC and vice versa. For 
instance, an index coding problem with 3 sources, as shown in Fig.|^ is a (3,1) instance in our model. 


B. Rate Region 

Having defined a network coding problem, we now define a network code and the network coding rate region. 

Definition 1: An (n, R) block code, with R = [n,..., tk, i?i,..., Rl] e K+', consists of a series of mutually 
independent sources Yp\ uniformly distributed in ^ = {1,..., [2”^*]}, and block encoders and decoders. 

i) The block encoders, one for each e g S’u, are functions that map a block of n source observations from 
all sources in S’je n ln(Tl(e)), and the incoming messages associated with the edges S'u n ln(Tl(e)), to one 
of different descriptions in = {0,1,..., 7?e - 1}, where r/e = [2"^'], 

: n n ^ e e 4/. (1) 

se<?j?’nIn(Tl(e)) ie<^’t,nIn(Tl(e)) 

a) The block decoders, one for each sink t e are functions 

4") : n n ll (2) 

eGln(t)nrf’(7 sGln(t)n<f’s' sG/3(t) 
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Denote by G the random message on edge e e (ou, which is the result of the encoding function fe^\ 


Further, we can define the probability of error for each sink f G ^ as 

p(n),err(R) ^ p ^ G m] 

and the maximum over these as 


fn),err/T>\ in),err 

(R) = mdppl 


(3) 


(4) 


Definition 2: The rate region of a network A, denoted as .^c(A), is the closure of the set of all achievable 
rate vectors R, where a rate vector R g .^c(A) is achievable if there exist a sequence of encoders {/(”) = 


[/i"V € <?]} and decoders e 5^]} such thatp(")’®''''(R) -> 0 as n 


An) 


00 . 


The rate region ^c{A) can be expressed in terms of the region of entropic vectors, T^, as in |[g, |[^. The 


discussion on FJ^ and its bounds is deferred to §II-C1 For the hyperedge MSNC problem, define a set ^ = 


{Ys^Ue\s G G (ou} with single letter random variables associated with sources and edges, respectively, and 
define N = \yK\ = K -\- L. Then, if we collect joint entropies of all non-empty subsets of ^ into a vector 
h = C 2^], we have h G TY. 


As will be shown in §II-C1 FJ^ is in the space of Note that the edge capacities, r = [Re\e G (ou], are extra 

variables associated with each edge. Therefore, we will consider the space in M^, where M = 2^ —1+L, L = \(ou\. 
We define i = 1, 3,4', 5 as network constraints representing source independence, coding by intermediate nodes, 
edge capacity constraints, and sink nodes decoding constraints respectively: 



= {h G 

(5) 


= {h G ; Auo„t(g)|(Y^ni„(a)UU^„nin(s)) = 0, p G 

(6) 

^4' 

= {[h^,r^]^ G M+ : i?e > A[/^,e G 

(7) 

-Sfs 

= {h G = 0,Vt G 

(8) 

3 = 

^1 n ^3, ^4'5 = ^4' n ,5^5 and H ,5^3 H ,5^4' n ^5- 

Note that we do 


not have ^2 constraints (which represent the coding function at each source) as in fT7| , due to our different 
notation with Ue = Yg if Tl(e) = s. Further, F^ and = 1,3,5 are viewed as subsets of with indexed 
by unconstrained, since they actually are in the space of ^4/ is also viewed as subset of M^, with 

the unreferenced dimensions (i.e. all non-singleton entropies) left unconstrained. The following extension of the 
rate region from m characterizes our slightly different rate region formulation <^c(A) for our slightly different 
problem. 


Theorem 1: The expression of the rate region of a network A is 


^c{A) = Proj^ ,^(con(rj:, n^is) n,5f4'5), 

where con{£g) is the conic hull of and Proj^ ^(^) is the projection of the set 
where r = [Re\e G Su] and w = [iT(Fs)|s g ,5^]. 


(9) 

on the coordinates 
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Proof: We present a sketch of the proof here and a detailed proof in Appendix First observe that the proof 
of Theorem 1 in (m can be extended to networks presented above, with hyperedges and intermediate nodes having 
direct access to sources. Some differences include: I) the hyperedge model potentially makes one edge variable 
connected with more than one node and thus be involved in more than one intermediate node constraint (ffs). 
Therefore, it may constrain more on the edge variable in the region of entropic vectors; II) the coding function 
for each intermediate (hyper)edge may encode some source edges with some other non-source edges together; III) 
the decoding at sink nodes may be a function of some source edges and non-source edges as well; IV) there is 
only one outgoing edge for each source and it carries the source variable itself. The differences will not destroy 
the essence of the proofs in (n). For the converse and achievability proof, we view the edge capacities as constant 
(recall that our rate vector include both source entropies and edge capacities), and then consider the converse and 
achievability of the associated source entropies, which becomes essentially the proof in HD- ■ 

While the analytical expression determines, in principle, the rate region of any network under network coding, it 
is only an implicit characterization. This is because F^ is unknown and even non-polyhedral for N > 4. Further, 
while is a convex cone for all A^, F^ is already non-convex by = 3, though it is also known that the closure 
only adds points at the boundary of FJ^. Thus, the direct calculation of rate regions from (|^ for a network with 4 or 
more variables is infeasible. On a related note, at the time of writing, it appears to be unknown by the community 
whether or not the closure after the conic hull is actually necessar}[^ in (|^, and the uncertainty that necessitates its 
inclusion muddles a number of otherwise simple proofs and ideas. For this reason, some of the discussion in the 
remainder of the manuscript will study a closely related inner bound to ^c(A) described in the following corollary. 
In all of the cases where the rate region has been computed to date these two regions are equivalent to one another. 

Corollary 1: The rate region ^c(A) of a network A is inner bounded by the region 

^*(A) = Proj^^^con(r;^) H (10) 

Proof: Clearly <^^ 4 / 5 ) 3 Proj^^^(con(rj;j H^is) 0 ^ 4 / 5 ). Next, observe that 

intersecting with ^13 is equivalent to requiring certain information inequalities (which are non-negative for all 
entropic vectors) to be identically zero, and a conic combination of such entropic vectors thus can only yield such 
an information inequality identically zero if the same information inequality was identically zero for each entropic 
vector. Hence con(r^ D^is) = con(r^) Dffis, and thus, Proj^ ,^(con(r5;j D^ 13 ) 0 ^ 4 / 5 ) = Proj^ ,^(con(r5;j) D 
^a)- This completes the proof. ■ 

Again, both <^c(A) and its closely related inner bound ^*(A) are not directly computable because they depend 
on the unknown region of entropic vectors and its closure. However, replacing PJ^ with finitely generated inner 
and outer bounds, as described in the following corollaries, transforms into a polyhedral computation problem, 
which involves applying some linear constraints onto a polyhedron and then projecting down onto some coordinates. 

^The closure would be unnecessary if = con(r^), i.e. if every extreme ray in F^ had at least one point along it that was entropic (i.e. 
in F^). At present, all that is known is that F^ has a solid core, i.e. that the closure only adds points on the boundary of F^. 
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Corollary 2: Let ^ c be some finite set of entropic vectors, then a polyhedral inner bound to the rate 
region is given by 

^c(A) ^ ^ Proj^ ,^(con(^) H^a). (11) 

Proof: It is clear that Proj^ ^(con(^ D ^ 13 ) 0 ^ 4 / 5 ) will be an inner bound to Proj^ ^(con(r^ D ^ 13 ) 0 ^ 4 / 5 ) 
and hence ^*(A). Furthermore, for such a finite set ^ Off 13 must also be a finite set, and hence con(^ D ffis) = 
con(^ nffis) is a closed polyhedral cone. Additionally, observe that every equality in ^13 can be viewed as setting 
a non-negative definite information inequality quantity to zero, and since every point in ^ must thus lie in only 
the non-negative half spaces these equalities generate, the extreme rays of con(^ fl ffis) must be those extreme 
rays of con(^) in ^ 13 , implying con(^ D ffis) = con(^) D ^ 13 . Putting these facts together we observe that 
the inner bound Proj^^^(con(^ D ffis) 0 ^ 4 / 5 ) = Proj^^^(con(^) Dffis 0 ^ 4 / 5 ) = Proj^^^(con(^) Pi^a)- ■ 

Similarly, polyhedral cones F^^ outer bounding the convex cone FJ^ yield polyhedral outer bounds to the rate 
region. 

Corollary 3: Let F^^ be a closed polyhedral cone that contains F^, then a polyhedral outer bound to the 
rate rate region is given by 

^c(A) C n ^a) (12) 

Proof: Since c c T*^n^n C C Thus, con(rj:, n C con(r°“‘n ifig) = 

n ^ 13 . Hence ^,{A) = Proj,_„(con(r;, n ^ 13 ) n ^ n ^a). ■ 

These corollaries inspire us to substitute P^ with such closed polyhedral outer and inner bounds r^*,and 
Pj^ = con(, 2 /), 32 / C PJ/ to P^, respectively, to obtain an outer and inner bound on the rate region: 

^out(A) = proj^ j^(r^‘n,5^A), (13) 

mn{A) = proj,,^(ri((nifA). (14) 

If ^out(A) = <^in(A), we know ^c(A) = <^*(A) = ^out(A) = <^in(A). Otherwise, tighter bounds are necessary. 

In this work, we will use ( p^ and ( p^ to calculate the rate region. Typically the Shannon outer bound F ^ and 
some inner bounds obtained from matroids, especially representable matroids, are used. We will briefly review the 
definition of these bounds in the next subsection, while details on the polyhedral computation methods with these 
bounds are available in (T4), ( 13 , (17), GD- 

C. Construction of bounds on rate region 

An introduction to the region of entropic vectors and the polyhedral inner and outer bounds we will utilize from 
it can be found in greater detail in |T^ . Here we briefiy review their definitions for accuracy, completeness, and 
convenience. 

1) Region of entropic vectors FJ^.* Consider an arbitrary collection X = [Xi, .. .^Xn] of N discrete random 
variables with joint probability mass function px- To each of the 2^ — 1 non-empty subsets of the collection of 
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random variables, := [Xi\i G sz/] with sz/ C { 1 ,..., N}, there is associated a joint Shannon entropy 
Stacking these subset entropies for different subsets into a 2^ — 1 dimensional vector we form an entropy vector 

h = [H{X^)\s^ C {1,..., X}, ^ ^ 0]. (15) 

By virtue of having been created in this manner, the vector h must live in some subset of , and is said to be 

entropic due to the existence of px- However, not every point in is entropic since, for many points, there 

does not exist an associated valid distribution px- All entropic vectors form a region denoted as It is known 
that the closure of the region of entropic vectors FJ^ is a convex cone pl] |. Elementary inequalities on Shannon 
entropies should form a fundamental outer bound on F^, named the Shannon outer hound Fjv- 

2) Shannon outer hound F x' We observe that elementary properties of Shannon entropies indicate that H{X^) 

is a non-decreasing submodular function, so that \/s^ ^ ^ {1, • • •, X}, ^ C N} 

H{X^) < H{X^) (16) 

X(Xc^u^) + X(Xc^n^) < H{Xc^)^H{X^). (17) 

Since they are true for any collection of subset entropies, these linear inequalities ( p^ , can be viewed as 
supporting half spaces for F^. 

Thus, the intersection of all such inequalities form a polyhedral outer bound Fat for FJ^ and F^, where 

C ^ 1 

h^us> + h^ns> < , Q) J 

This outer bound F^v is known as the Shannon outer hound, as it can be thought of as the set of all inequalities 
resulting from the positivity of Shannon’s information measures among the random variables. While F 2 = FJ and 
F 3 = Fg, FJ^ C Fat for all X > 4 |[^, and indeed it is known ||^ that FJ^ is non-polyhedral for X > 4. 

The inner bounds on F^ we consider are based on representable matroids. We briefly review the basic deflnitions 
of matroids and representable matroids. 

3) Matroid basics: Matroid theory | [24| is an abstract generalization of independence in the context of linear 
algebra and graphs to the more general setting of set systems. There are numerous equivalent deflnitions of matroids, 
however, we will present the deflnition of matroids utilizing rank functions as this is best matched to our purposes. 

Definition 3: A set function on a ground set tm : 2=^ ^ {0,..., |^|}, is a rank function of a matroid M 
if it obeys the following axioms: 

1 ) Cardinality: rM(^) < 1^1; 

2) Monotonicity: if ^ c ^ c ^ then ru{^) < 

3) Submodularity: if c ^ then rM(^u^) +rM(^n^) < ru{^) 

A subset with rank function = \^\ is called an independent set of the matroid. Though there are many 

classes of matroids, we are especially interested in one of them, representable matroids, because they can be related 
to linear codes to solve network coding problems as discussed in (13 eg 
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4) Representable matroids: Representable matroids are an important class of matroids which connect the inde¬ 
pendent sets to the notion of independence in a vector space. 

Definition 4: A matroid M with ground set ^ of size \^\ = N and rank = r is representable over 

a field F if there exists a matrix A g such that for each set ^ c ^ the rank r^i^) equals the linear 
rank of the corresponding columns in A, viewed as vectors in F"’. 


Note that, for an independent set the corresponding columns in the matrix A are linearly independent. There 
has been significant effort towards characterizing the set of matroids that are representable over various field sizes, 
with a complete answer only available for fields of sizes two, three, and four. For example, a matroid M is binary 
representable (representable over a binary field) iff it does not have the matroid f/ 2,4 as a minor. Here, a minor is 
obtained by series of operations of contraction and deletion | [24| . Uk,N is the uniform matroid on the ground set 
^ = {1,..., A^} with independent sets ^ equal to all subsets of A^} of size at most k. For example, f/ 2,4 

has as its independent sets 

^ = {0,1,2,3,4,{1,2},{1,3},{1,4},{2,3},{2,4},{3,4}}. (18) 


Another important observation is that the first non-representable matroid is the so-called Vdmos matroid, a well 
known matroid on ground set of size 8. That is to say, all matroids are representable, at least in some field, for 
N <7. 

5) Inner hounds from representable matroids: Suppose a matroid M with ground set ^ of size \^\ = N and 
rank ru{^) = r is representable over the finite field F^ of size q and the representing matrix is A G F^^^ such 
that r\^{^) = rank(A:^^), C the matrix rank of the columns of A indexed by Let F^ be the conic 
hull of all rank functions of matroids with N elements and representable in ¥q. This provides an inner bound 
C FJ^, because any extremal rank function of F^ is by definition representable and hence is associated with a 
matrix representation A G F^^^, from which and r random variables u uniformly distributed in F^, we can create 
the random variables 

[Xi,... ,Xn] = uA, u ^ Uniform(Fg), (19) 


whose elements have joint entropies = rM(^) log 2 C Hence, all extreme rays of F^ are entropic, 

and F^ C F^. Further, if a vector in the rate region of a network is (a projection of) an Fg-representable matroid 
rank, the representation A can be used as a linear code to achieve that rate vector, and this code is denoted a basic 
scalar F^ code. For an interior point in the rate region, which is the conic hull of projections of F^-representable 
matroid ranks, the code to achieve it can be constructed by time-sharing between the basic scalar codes associated 
with the ranks involved in the conic combination. This code is denoted a scalar Fg code. Details on construction 
of such a code can be found in (B) and d). 

One can further generalize the relationship between representable matroids and entropic vectors established by 
(19). Suppose the ground set = {!',..., N'} and a partition ^ = {1,...,X}. We define a partition mapping 


TT : such that , and 7r(i') D 7r(j') = ,j' G Jf' 7^ /. That is, the set Jf' is 
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partitioned into N disjoint sets. Suppose the variables associated with are Xi/,... Now we define for 

n G ^ the new vector-valued random variables = [Xi'\i' G 7r“^(n)]. The associated entropic vector will have 

entropies = rM(UnG^7r“^(n)) log 2 g, ^ and is thus proportional to a projection of the original rank 

vector r keeping only those elements corresponding to all elements in a set in the partition appearing together. 
Thus, such a projection of T^, forms an inner bound to TJ^, which we will refer to as a vector representable 
matroid inner bound T^ As N' oo, T^ ^ is the conic hull of all ranks of subspaces on F^. The union over 
all field sizes for T^ ^ is the conic hull of the set of ranks of subspaces. Similarly, if a vector in the rate region of 
a network is (a projection of) a vector Fg-representable matroid rank, the representation A can be used as a linear 
code to achieve that rate vector, and this code is denoted as a basic vector F^ code. The time-sharing between such 
basic vector codes can achieve any point inside the rate region (ID, GD 

6 ) Dimension function of linear sub space arrangements: As stated above, T^ becomes a tighter and tighter 
inner bound on T^ as X' ^ oo. This considers the increase in dimension but does not consider the fields other 
than ¥q. Actually, if we consider all possible F^ fields and let N' oo, we will get the inner bound associated with 
all linear codes, denoted by which is tighter than T^ ^ for a fixed ¥q. Specifically, consider a collection 

of N linear vector subspaces V = (Yi,..., Vat) of a finite dimensional vector space, and define the set function 
d : 2^ ^ N+, where d(^) = dim ^ ^ {1^ • • • is the dimension of the vector space 

generated by the union of subspaces indexed by For any collection of subspaces V, the function d is integer 
valued, and obeys monotonicity and submodularity. Additionally, for every subspace dimension function d, there is 
an associated entropic vector. Indeed, one can place the vectors forming a basis for each Vi, over all i, side by side 
into a matrix A, which when utilized in ( p^ , will yield random subvectors having the desired entropies. Thus, the 
conic hull of dimensions of linear subspace arrangements forms an inner bound on T^, we denote it by 

Integrality, monotonicity, and submodularity are necessary but insufficient for for a given set function d : 2^ —> 
N+ to be dimension function of subspace arrangements. That is, there exist additional inequalities that are necessary 
to describe the conic hull of all possible subspace dimension set functions. As discussed in p0| , Ingleton’s inequality 
together with the Shannon outer bound r 4 , completely characterizes 
For X = 5 subspaces found 24 new inequalities in addition to the Ingleton inequalities that hold, and prove 
this set is irreducible and complete in that all inequalities are necessary and no additional non-redundant inequalities 
exist. For X > 6, there are new inequalities from X — 1 to X, and remains unknown. 

All the bounds discussed in this section could be used in and ( p^ to calculate bounds on rate regions for 
a network A. If we substitute the Shannon outer bound F n into GD, we get 

^,(A) = proj,,„(rArnifA)- (20) 

Similarly, we substitute the representable matroid inner bound the vector representable matroid inner bound 
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^ and the linear inner bound into ([T^, to obtain 



( 21 ) 


( 22 ) 


(23) 


(24) 


We will present the experimental results utilizing these bounds to calculate the rate regions of various networks 
in ^V] However, before we do this, we will first aim to generate a list of network coding problems to which we 
may apply our computations and thereby calculate their rate regions. In order to tackle the largest collection of 
networks possible in this study, in the next section we will seek to obtain a minimal problem description for each 
network coding problem instance, removing any redundant sources, edges, or nodes. 


III. Minimality Reductions on Networks 


Though in principle, any network coding problem as described in ^ forms a valid network coding problem, such 


a problem can include networks with nodes, edges, and sources which are completely extraneous and unnecessary 
from the standpoint of determining the rate region. To deal with this, in this section, we show how to form a network 
instance with equal or fewer number of sources, edges, or nodes, from an instance with extraneous components. 
We will show the rate region of the instance with the extraneous components is trivial to calculate from the rate 
region of the reduced network. Network coding problems without such extraneous and unnecessary components 
will be called minimal. We first define a minimal network coding problem, then show, via Theorem]^ how to map 
a non-minimal network to a minimal network, and then form the rate region of the non-minimal network directly 
from the minimal one. 

Definition 5: An acyclic network instance A = {y, y, S’, P) is minimal it obeys the following constraints: 
Source minimality: 

(Cl) all sources cannot be only directly connected with sinks: Vs g Hd(Out(s)) n ^ 7 ^ 0; 

(C2) sinks do not demand sources to which they are directly connected: \/s e y, t e y/\it e Hd(Out(s)) 
then s ^ P{t)] 

(C3) every source is demanded by at least one sink: Ms e y,3t e y such that s g P{t) ; 

(C4) sources connected to the same intermediate node and demanded by the same set of sinks should be 


merged: $s,s' g ^ such that Hd(Out(s)) = Hd(Out(s')) and 7(5) = 7(5'), where 7(s) = {t g y\s g 


Node minimality: 

(C5) intermediate nodes with identical inputs should be merged: $k,l eW such that ln{k) = In(/); 

(C 6 ) intermediate nodes should have nonempty inputs and outputs, and sink nodes should have nonempty 
inputs: g t g 5^, In(^) ^ 0, Ou.t{g) ^ 0, In(t) ^ 0; 
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Edge minimality. 

(C7) all hyperedges must have at least one head: $e e S’ such that Hd(e) = 0; 

(C8) identical edges should be merged: e' e S with Tl(e) = Tl(e'), Hd(e) = Hd(e'); 

(C9) intermediate nodes with unit in and out degree, and whose in edge is not a hyperedge, should be 
removed: $e,e' e S,g eW such that In(^) = e, Hd(e) = g, Out(^) = e'\ 

Sink minimality. 

(CIO) there must exist a path to a sink from every source wanted by that sink: Vt g c cr(t), where 

(j{t) = {k e a path from k to t}\ 

(Cll) every pair of sinks must have a distinct set of incoming edges: ^ j, In(t) / In(t'); 

(C12) if one sink receives a superset of inputs of a second sink, then the two sinks should have no common 
sources in demand: If In(t) c In(t'), then ^{t) n = 0; 

(C13) if one sink receives a superset of inputs of a second sink, then the sink with superset input should not 
have direct access to the sources that demanded by the sink with subset input: If In(t) c in(t') then 
t' ^ Hd(Out(5)) for all 5 G /3(t). 

Connectivity. 

(C14) the direct graph associated with the network A is weakly connected. 


To better highlight this definition of network minimality, we explain the conditions involved in greater detail. The 
first condition (Cl)| requires that a source cannot be only directly connected with some sinks, for otherwise no sink 


needs to demand it, according to |(C2)| and |(C10)| Therefore, this source is extraneous. The condition |(C2)| holds 
because otherwise the demand of this sink will always be trivially satisfied, hence removing this reconstruction 
constraint will not alter the rate region. Note that other sources not demanded by a given sink can be directly 


available to that sink as side information (e.g., as in index coding problems), as long as condition (C13) is satisfied. 


The condition (C3) indicates that each source must be demanded by some sink nodes, for otherwise it is extraneous 


and can be removed. The condition (C4) says that no two sources have exactly the same paths and set of demanders 
(sinks requesting the source), because in that case the network can be simplified by combining the two sources 


as a super-source. The condition (C5) requires that no two intermediate nodes have exactly the same input, for 


otherwise the two nodes can be combined. The condition |(C6)| requires that no nodes have empty input except the 
source nodes, for otherwise these nodes are useless and extraneous from the standpoint of satisfying the network 


coding problem. The condition (C7) requires that every edge variable must be in use in the network, for otherwise 


it is also extraneous and can be removed. The condition (C8) guarantees that there is no duplication of hyperedges. 


for otherwise they can be merged with one another. The condition (C9) says that there is no trivial relay node with 
only one non-hyperedge input and output, for otherwise the head of the input edge can be replaced with the head 


of the output edge. The condition |(C10)| reflects the fact that a sink can only decode the sources to which it has at 
least one path of access, and any demanded source not meeting this constraint will be forced to have entropy rate 


of zero. The condition (Cll) indicates the trivial requirement that no two decoders should have the same input. 
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for otherwise these two decoders can be combined. The condition |(C12)| simply stipulates that implied capabilities 
of sink nodes are not to be stated, but rather inferred from the implications. In particular, if In(f) C In(f'), and 
j3{t) n /3(f') 7 ^ 0, the decoding ability of /3(f) is implied at t'\ pursuing minimality, we only let t' demand extra 
sources, if any. The condition |(C13)| is also necessary because the availability of s is already implied by having 
access to In(f), hence, there is no need to have direct access to s. 

We next show that the rate region of the network with extraneous components can be easily derived from the 


network without extraneous components, and vice versa. Following the same order of the constraints (C1) -(C14) 


we give the actions on each reduction and how the rate region of the network with those extraneous components 
can be derived. 


Theorem 2: Suppose a network instance A = with rate region and bounds I e 

{c,*,q,{s,q),o}, is a reduction from another network A' = with rate region bounds 

e {c,*,q, {s,q),o}, by removing one of the redundancies specified in |(C1)f[(C14)| in Def.Then, 
by defining R\^ = Proj^^R to be the projection of R excluding coordinates associated with y, and be 
the source rate of s, we have the following. 

Source minimality. 

(Dl) If 3s' e y, Hd(Out{s')) n = 0, A will be A' with s' removed and 


:= {R|R\s' e ^;(A), Us' =0} Vf e {c, *, q, (s, q), o} (25) 

if 3t' e y such that s g p{t') and s ^ In(f'), while 

^i(A') := {R|R\s e ^i(A), Ws > 0} \/l e {c,*,q,{s,q),o} (26) 

Otherwise. Furthermore, 


^i(A) = Proj\g^;(A') Ml e {c,*,q,{s,q),o}. (27) 

(D2) if 3s' G y,t' G y, such that t' g Hd(Out(s')) and s' g /3(f'), A will be A' with In(f) = In(f') \ s' and 
/3(f) = /3(f') \ s'. Further, .^;(A') = ^[{A) for all I e {c,*,q, {s,q),o}. 

(D3) if 3s' G y, such that Vf' g y, Yg' ^ /3(f'), A will be A' with removal of the redundant source s' and 

^,(A') := {R|R\,, G^i(A),iJ(y,/) >0} Ml € {c,*,q,{s,q),o} (28) 

and 

yi{A) =Proy^,yi{A') Ml e {c,*,q,{s,q),o}. (29) 

(D4) if 3s,s' G y such that Hd(Out(s)) = Hd(Out(s')) and 7 (s) = 7 (s'), A will be A' with sources s,s' 
merged and 


yi{A') = {R|[R(;,,,,},F(n) + F(y,,)]^ e.^i(A)} Ml G {c,yq,o}. 


(30) 
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(Cl) 


source S 3 does not connected 


(C2) 


sink ts has direct access to 


(C3) 


source Y 3 is not demanded 


with any intermediate node, and Y 2 , the demand of I 2 is trivially by any sink, and thus is redundant, 
thus is extraneous. satisfied and thus ts is redundant. 



(C4) 


sources Yi , Y 3 have exactly the 


(C5) 


node gi,g 2 have same input. 


same output and demanders, and thus and thus can be combined, 
can be combined. 


(C 6 ) 


node 5 ^ 3 , 5^4 and sink ti have 
empty input/ output, and thus are re¬ 
dundant. 



(C7) edge U 2 is not connected 
to any other nodes, and thus is 
redundant. 



(C 8 ) edges U 2 ^U 3 have exactly 
the same input and output nodes, 
and thus can be combined. 


(C9) node gi/ has exactly one 
input and one output, and they can 
be combined. 



U3 


CIO: sink ^3 has no access to si but 
demands Ifi, so the only way to sat¬ 
isfy it is Si is sending no information. 



sinks ti,t 2 have exactly 


the same input and thus can be 
combined into one sink node. 



(C12) 


ti decodes Y 2 from Ui, 
hence t 2 also can decode Y 2 , thus 
there is no need to list Y 2 in f 3 {t 2 ) 



(C13) 


ti decodes Y 2 from Ui, 
thus t 2 also can decode Y 2 , thus 
there is no need to keep direct 
access of t 2 to Y 2 . 




Y3 


(C14) each connected compo¬ 
nent can be viewed as a separate 
network instance. 


Fig. 5: Examples to demonstrate the minimality conditions 


(Cl) 


(C14) 
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i.e., replace H{Ys) in ^i{A) with H{Ys) + H{Ys<) to get ^((A'), I e {*,q,o}. Furthermore, 

,^i(A) = {R\{s} |R G = 0} e {c,*,q,{s,q),o} . (31) 

Node minimality. 

(D5) If 3k', f e such that In(A:') = In(j'), A will be A' with k',j' merged so that In(fc) = In(A:') = 
In(j'),Out(fc) = Out(fc')uOut(j'), and Further, = S^i{N) for all I e {c, *,q, {s,q),o}. 

(D 6 ) If 3g' e such that 111 ( 5 ) = 0. of Out( 5 ) = 0, A will be A' with removal of the redundant node(s) g' 

and^i(A') = ^;(A) for all I e {c,*,q,{s,q),o}. 

Similarly, if 3t' e SA such that In(f ) = 0, A will be A' with removal of the redundant node(s) t' 
and the deletion of any sources it demands, = {R|R\^(t/) e ^i{A),los = OVs e and 

^i{A) = Proj\^(t,),^;(A')for all I e {c, *, q, (s, q), o}. 

Edge minimality: 

(D7) If 3e' e S' such that Hd(e') = 0, A will be N with removal of edge e', 

,!^;(A') = {R|R\e' e ^l{A),Re' > 0 }, (32) 

and^i(A) = Proj\g„^i(A') for all I e {c,*,q,{s,q),o}. 

(D 8 ) If 3e, e' G S' with Tl(e) = Tl(e'), Hd(e) = Hd(e'), A will be A' with edges e, e' merged as e and 

= {R|[Rue,e'}>^e + i?e']^ e,!^KA)}; V( G {c, 5, o} (33) 

i.e., replace Re in ,^*(A) with Re + Re' to get ^*(A'). Furthermore, 

,^,(A) = {[R(’{,^^,^,Re]^|Re,^,(A'),Re'= 0 } V(G{c,>i, 5 ,(s, 5 ),o}. (34) 

(D9) If 3e,e' G S',g' G 'S' such that 111 ( 5 ') = e. Hd(e) = 5 ', Out( 5 ') = e', then A will be A' with the node 5 ' 
removed and a new edge Ci replacing e, e' by directly connecting Tl(e) and Hd(e'). Further, 

^;(A') = {R|[R^{,,,,},niin{Re,-Re'}]^ e ^;(A)} ; (35) 

i.e., replace Re in Sli{A) with min{ii’e, Re'} to get ^;(A'). Accordingly, 

m{A) = {[It(^^^^,y,mm{Re,Re'}f |R G m{A') } . (36) 

for all I G {c, *,q, {s,q),o}. 

Sink minimality: 

(DIO) If 3t' G S'', s' G y, such that s' g /3{t') but s' ^ cr(t'), then A will be A' with s' deleted, 

y{A') = {R|Rv e m{A),H{Ye') = 0}, (37) 

and 

,^,(A) =Proj\,,^i(A') (38) 

for all I G {c, *,q, {s,q),o}. 
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(Dll) If 3t,t' e ^ t', such that In(t) = In(t'), then A will be A' with sinks t,t' merged and ^i{A') = 
^i{A) for all I e {c, *, ( 5 , q),o}. 

(D12) If t' such that In(t) c In(t') and ^{t) n /3(t') ^ 0, then A will be A' with removal of f3{t) n from 
P{t') and^/(A') =^/(A) for all I e {c,^,q,{s,q),o}. 

(D13) If t', 5 ' G I3{t) such that In(t) c In(t') and t' e Hd(Out(s')), then A will be A' with removal of s' from 

In(t') and ^i{A') = ^i{A) for all I e {c, *, g, (s, g), o}. 

Connectivity: 

(D14) if A' is not weakly connected and Ai,A 2 are two weakly disconnected components, then ^i{A') = 
^z(Ai) X ^z(A2) for all I e {c, *, g, (s, g), o}. 


Proof: In the interest of conciseness, for all but (D4) and (D 8 ) we will only briefly sketch the proof for the 
expressions determining ^*(A') from ^^{A), as the map in the opposite direction and the other rate region bounds 
follow directly from parallel arguments. 


(Dl) holds because s' is not communicating with any nodes other than possibly sinks. If there is a sink that 


demands it that does not have direct access to it, then this sink can not successfully receive any information from 
it, since s' does not communicate with any intermediate nodes. Hence, in this case oos' = 0 and every other rate 
is constrained according to <^*(A) because the remainder of the network has no interaction with s'. Alternatively, 
if every sink that demands s' has direct access to it, any non-negative source rate can be supported for s', and the 
remainder of the network is constrained as by ^*(A) because no other part of the network interacts with s'. 


(D2) holds because the demand of s' at sink t' is trivially satisfied if it has direct access to s'. The constraint 


has no impact on the rate region of the network. 


In (D3) if a source is not demanded by anyone, it can trivially support any rate. 


When two sources have exactly the same connections and are demanded by same sinks as under |(D4)| they can 
be simply viewed as a combined source for with I G {c, *, g, o}, since the exact region and these bounds enable 
simple concatenation of sources. Since the source entropies are variables in the rate region expression, it is equivalent 
to make s as the combined source, which since the previous sources were independent, will have an entropy which 
is the sum of their entropies. Moving from ^i{A') to ^i{A) is then accomplished for any I G {c, *, g, ( 5 , g), 0 } by 
observing that A can be viewed as A' with ujg' =0. 

An intermediate node can only utilize its input hyperedges to produce its output hyperedges, hence when two 
intermediate nodes have the same input edges, their encoding capabilities are identical, and thus for pursuing 
minimality of representation of a network, these two nodes having the same input should be represented as one 


node. Thus, (D5) is necessary and the merge of nodes with same input does not impact the coding on edges or the 
rate region, as the associated constraints • 


If the input or output of an intermediate node is empty, as in (D 6 ) it is incapable of affecting the capacity region. 


If, as in the second case covered by (D 6 ) the input to an sink node is empty, any sources which it demands can 


only be reliably decoded if they have zero entropy. 
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(D7) is clear because an edge to nowhere can not effect the rest of the capacity region and is effectively 


unconstrained itself. 


(D8) can be shown as follows. If ^*(A) is known and when edge e in A is represented as two parallel edges 


e, e' so that the network becomes A', then the constraint on e, e' in A' is simply to make sure the total capacity 
Re + Re' can allow the information to be transmitted from the tail node to head nodes. Simple concatenation of 
the messages among the two edges will achieve this for those bounds I G {c, *, g, o} allowing such concatenation. 
Therefore, replace the Re in ^/(A) with Re^Re' will obtain the rate region ^z(A') for any I G {c, *, g, o}. Moving 
from to ^i{A) is accomplished by recognizing that A is effectively A' with R'^ = 0. 


Under the condition in (D9) an intermediate node g' has exactly one input edge e and exactly one output 
hyperedge e', and the input e is an edge (i.e. g' is its only destination). The rate coming out of this node can be 
no larger than the rate coming in since the single output hyperedge must be a deterministic function of the input 
edge. It suffices to treat these two edges as one hyperedge connecting the tail of e to the head of e' with the rate 
the minimum of the rates on the two links. 

If a sink demands a source that it does not have access to, the only way to satisfy this network constraint is the 


source entropy is 0. Hence, (DIO) holds. The removal of this redundant source does not impact the rate region of 
the network with remaining variables. 


(Dll) similar to TO observes that two sink nodes with same input yield the exact same constraints as 


with the two sink nodes merged. 


(D12) is easy to understand because the decoding ability of ^(t) at sink node t is implied by sink t'. The 


non-necessary repeated decoding constraints will not affect the rate region for this network. 


(D13) similar to (D12) observes that the ability of t to decode s' implies that t' can decode it as well, and 


hence, adding or removing the direct access to s' at t' will not affect the rate region. 

|(D14)| is obviously true since the weakly disconnected components can not infiuence each others rate regions. ■ 
Fig.[^ contains examples illustrating these reductions. In general, we can define a minimality operator A = 


minimal(A') on networks, which checks the minimality conditions (C1) -(C14) on A' one by one, in the order 
|(C1)[ |(C2)| |(C6)[ |(C5)[ |(C3)| |(C4)[ |(C7)f|(C14)| If any of the conditions encountered is not satisfied, the network 
is immediately reduced it according to the associated reduction in Theorem and the resulting reduced network 
is checked again for minimality by starting again at condition (Cl)[ if needed, until all minimality conditions are 
satisfied. Furthermore, define the associated rate region operator ^^{A') = minimalA'^A('^*(A)) which moves 
through each of the reduction steps applied by minimal (A') to the network A' in reverse order, utilizing the 
expression for the rate region change under each reduction, thereby obtaining the rate region of A' from A. 
Accordingly, let ^*(A) = minimalA'^A('^*(A')) be the rate region operator which moves through each of the 
reduction steps applied by minimal (A') to the network A' in order, utilizing the expression for the rate region 
change under each reduction, thereby obtaining the rate region of A from A'. This network minimality operator and 
its associated rate region operators will come in use later in the paper. However, we next discuss the enumeration 
of minimal networks of a particular size. 
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Fig. 6: A demonstration of the equivalence between network coding problems via isomorphism: networks I and II are 
equivalent because II can be obtained by permuting 11,12. However, network III is not equivalent to I or II, because the 
demands at the sinks do not reflect the same permutation of Yi , 11 cis is necessary on the source side. 


IV. Enumeration of Non-isomorphic Minimal Networks 

Even though the notion of network minimality (^In|) reduces the set of network coding problem instances by 
removing parts of a network coding problem which are inessential, much more needs to be done to group network 
coding problem instances into appropriate equivalence classes. Although we have to use label sets to describe the 
edges and sources in order to specify a network coding problem instance (identifying a certain source as source 
number one, another as source number two, and so on), it is clear that the essence of the underlying network coding 
problem is insensitive to these labels. Eor instance, it is intuitively clear that the first two problems depicted in 
Eig. should be equivalent even though their labeled descriptions differ, while the third problem should not be 
considered equivalent to the first two. 

In a certain sense, having to label network coding problems in order to completely specify them obstructs 
our ability to work efficiently with a class of problems. This is because one unlabeled network coding problem 
equivalence class typically consists of many labeled network coding problems. In principle, we could go about 
investigating the unlabeled problems by exhaustively listing labeled network coding problem obeying the minimality 
constraints, testing for equivalence under relabeling of the source and node or edge indices, and grouping them 
together into equivalence classes. 

However, listing networks by generating all variants of the labeled encoding becomes infeasible rapidly as the 
problem grows because of the large number of labeled networks in each equivalence class. As a more feasible 
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alternative, it is desirable to find a method for directly cataloguing all (unlabeled) network coding problem equiv¬ 
alence classes by generating exactly one representative from each equivalence class directly, without isomorphism 
(equivalence) testing. 

In order to develop such a method, and to explain the connection between its solution and other isomorphism-free 
exhaustive generation problems of a similar ilk, in this section we first formalize a concise method of encoding 
a network coding problem instance in ^IV-A With this encoding in hand, in ^IV-B the notion of equivalence 
classes for network coding problem instances can be made precise as orbits in this labeled problem space under an 
appropriate group action. The generic algorithm Leiterspiel p4| , p5| , for computing orbits within the power set 
of subsets of some set ^ on which a group G acts, can then be applied, together with some other standard orbit 
computation techniques in computational group theory | [36| , | [37| , in order to provide the desired non-isomorphic 
network coding problem list generation method in |IV-C[ 


A. Encoding a Network Coding Problem 

Though, as is consistent with the network coding literature, we have thus far utilized a tuple A = (^, /3) 

to represent a network instance, this encoding proves to be insufficiently parsimonious to enable easy identification 
of equivalence classes. As will be discussed later, the commonly used node representation of a network, a key 
component of the A = (^,^, ^,^,/3) encoding, unnecessarily increases the complexity of enumeration. Hence, 
we represent a network instance in an alternate way for enumeration. Specifically, a network instance with K sources 
and L edges that obeys the minimality conditions (C1-C14) is encoded as an ordered pair (^, W) consisting of 
a set of edge definitions C {(i, s^)\i g {K -1- 1,..., AT -1- L}, C {1,..., AT -f- L} \ {i}, \s^\ > 0}, 
and a set W of sink definitions W C {(i, s^)\i G {1,..., K}, C {1,..., if + A} \ {i}}. Here, the sources are 
associated with labels {1,..., if} and the edges are associated with labels {if + 1,..., if -|- A}. Each (i, G ^ 
indicates that the edge i G is encoded exclusively from the sources and edges in and hence represents the 
information that = In(Tl(i)). Furthermore, each sink definition (i, ^) G W represents the information that there 
is a sink node whose inputs are and which decodes source % as its output. Note that there are A non-source 


edges in the network, each of which must have some input according to condition |(C6)| We additionally have the 
requirement that \^\ — L, and, to ensure that no edge is multiply defined, we must have that if (i, ^) and (i', ^') 
are two different elements in then i i'. As the same source may be decoded at multiple sinks, there is no 
such requirement for W. 

As is illustrated in Figures |7^ and this edge-based definition of the directed hypergraph included in a network 
coding problem instance can provide a more parsimonious representation than a node-based representation, and as 
every edge in the network for a network coding problem is associated with a random variable, this representation 
maps more easily to the entropic constraints than the node representation of the directed acyclic hypergraph 
does. Additionally it is beneficial because it is guaranteed to obey several of the key minimality constraints. In 
particular, the representation ensures that there are no redundant nodes (C5) (Cll)| since the intermediate nodes 
are associated directly the elements of the set {^| (i, G <^} and the sink nodes are associated directly with 
















24 


{j^l 3i, G Representing as a set (rather than a multi-set) also ensures that (C8) is always obeyed, 

since such a parallel edge would be a repeated element in 


B. Expressing Network Equivalence with a Group Action 

Another benefit of the representation of the network coding problem as the ordered pair (^, #^) is that it enables 
the notion of network isomorphism to be appropriately defined. In particular, let G := x S{k+i,...,k+l} 

be the direct product of the symmetric group of all permutations of the set { 1 , 2 ,..., AT} of source indices and the 
symmetric group of all permutations of the set {if + 1 ,..., AT + L} of edge indices. The group G acts in a natural 
manner on the elements of the sets of edge and sink definitions. In particular, let tt G G be a permutation in 

G, then the group action maps 

7r((i,j^)) i-G (7r(i),7r(j^)) (39) 

with the usual interpretation that 7 r(^) = {7 r{j)\j G ^}. This action extends to an action on the sets ^ and W 
in the natural manner 

7r(^) i-G- {7r((i, ^)) I (i, ^) G . (40) 

This action then extends further still to an action on the network via 

(41) 

Two networks (<^i,#i) and (<^ 2 ,^) are said to be isomorphic, or in the same equivalence class, if there is 
some permutation of tt G G such that 7 r((c^i, # 1 )) = (^ 2 , # 2 )- In the language of group actions, two such pairs 
are isomorphic if they are in the same orbit under the group action, i.e. if (^ 2 ,^) ^ { 7 r((c^i, # 1 )) \'k G G} =: 
^(^ 1 ,^ 1 ) • In other words, the equivalence classes of networks are identified with the orbits in the set of all valid 
minimal problem description pairs under the action of G. 

We elect to represent each equivalence class with its canonical network, which is the element in each orbit that 
is least in a lexicographic sense. Note that this lexicographic (i.e., dictionary) order is well-defined, as we can 
compare two subsets and by viewing their members in increasing order (under the usual ordering of the 
integers {!,...,A-fAr}) and lexicographically comparing them. This then implies that we can lexicographically 
order the ordered pairs (i, according to (i, > (j, ^') if j < i ox i = j and under this lexicographic 

ordering. Since the elements of ^ and W are of the form (i,^), this in turn means that they can be ordered in 
increasing order, and then also lexicographically compared, enabling comparison of two edge definition sets ^ and 
or two sink definition sets W and W'. Finally, one can then use these orderings to define the lexicographic order 
on the network ordered pairs (c^, ^). The element in an orbit which is minimal under this lexicographic 

ordering will be the canonical representative for the orbit. 

A key basic result in the theory of group actions, the Orbit Stabilizer Theorem, states that the number of elements 
in an orbit, which in our problem is the number of networks that are isomorphic to a given network, is equal to the 
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ratio of the size of the acting group G and its stabilizer subgroup of any element selected from the orbit: 






:={7reG K((^, ^)) = (^, } (42) 


Note that, because it leaves the sets of edges, decoder demands, and topology constraints set-wise invariant, the 
elements of the stabilizer subgroup also leave the set of rate region constraints ([^, ([^, Q, ^ invariant. 

Such a group of permutations on sources and edges is called the network symmetry group, and is the subject 
of a separate investigation p^ , GD- This network symmetry group plays a role in the present study because. 


as depicted in Figures [7^ and [7^ by the orbit stabilizer theorem mentioned above, it determines the number of 
networks equivalent to a given canonical network (the representative we will select from the orbit). 


In particular. Fig. 7a shows the orbit of a (2, 2) network (^, ^) whose stabilizer subgroup (i.e., network symmetry 
group) is simply the identity, and hence has only one element. In this instance, the number of isomorphic labeled 
network coding problems in this equivalence class is then |G| = |S{i 2 } x S{ 3 ^ 4 } | = 4 in the edge representation, as 
shown at the left. Even this tiny example demonstrates well the benefits of encoding a network coding problem via the 
more parsimonious representation vs. the encoding via the node representation hypergraph and the 

sink demands Namely, because the size of the group acting on the node representation is x S^c,d,ej,g} I = 

240, and, as the stabilizer subgroup in the node representation has the same order (1), the number of isomorphic 
networks represented in the node based representation is 240. 

By contrast. Fig. 7b shows the orbit of a (2, 2) network (^, W) whose stabilizer subgroup (i.e., network symmetry 
group) is the largest possible among (2, 2) networks, and has order 4. In this instance, the number of isomorphic 
labeled network coding problems in this equivalence class is = 1 in the edge representation. The stabilizer 

subgroup in the node representation has generators ({(a, 6)(d,/)(e, ^)}, {(d, e)(/, ^)}), which has the same order 
of 4, and hence there are ^ = 60 isomorphic network coding problems to this one in the node representation. 


C. Network Enumeration/Listing Algorithm 

Formalizing the notion of a canonical network via group actions on the set of minimal (^, #^) pairs enables one 
to partly develop a method for directly listing canonical networks based on techniques from computational group 
theory. 

To solve this problem we can harness the algorithm Leiterspiel, loosely translated snakes and ladders p4| , p5| , 
which, given an algorithm for computing canonical representatives of orbits, i.e., transversal, on some finite set 
^ under a group G and its subgroups, provides a method for computing the orbits on the power set = 

|^|=i}of subsets from ^ of cardinality i, incrementally in i. In fact, the algorithm can also list 
directly only those canonical representatives of orbits for which some test function / returns 1, provided that the 
test function has the property that any subset of a set with / = 1 also has / = 1. This test function is useful for 
only listing those subsets in ^i{^) with a desired set of properties, provided these properties are inherited by 
subsets of a superset with that property. 
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Subgroup of S{a,6} X S{c,d,e,f,g} 
_stabilizing (V, ^) = (()) 


Subgroup of S{i^2} X S{3^4} stabilizing {G,T) = (()) 


Isomorphs in Node Representation (240) 

_lsomorphs in Edge and Sink 
Definition Representation (4) 


(a) All isomorphisms of a (2, 2) network with empty symmetry group 
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# 

Q 
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1 

{(3,{1,2}),(4,{1,2})} 

{(1,{2,3}),(1,{2, 4}),(2,{1,3}),(2,{1,4})} 


Subgroup of S{i, 2 } X S| 3 ^ 4 } stabilizing {Q,T) = S{i^ 2 } x S{ 3 ^ 4 } 


The stabilizer subgroup is of order 4 


(b) All isomorphisms of a (2, 2) network with full symmetry group 

Fig. 7: Examples o/(2, 2) networks with all edge isomorphisms (left) and all node isomorphisms (right). The instance indices 

are marked by # and the labels are marked by 1. 
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To see how to apply and modify Leiterspiel for network coding problem enumeration, let ^ be the set of possible 
edge definitions 

SC := G + + + (43) 

For small to moderately sized networks, the orbits in ^ from G and its subgroups can be readily computed with 
modern computational group theory packages such as GAP p7| or PERMLIB p6| . Leiterspiel can be applied to 
first calculate the non-isomorphic candidates for the edge definition set as it is a subset of ^ with cardinality L 
obeying certain conditions associated with the definition of a network coding problem and its minimality (c.f. Cl- 
C14). Next, for each non-isomorphic edge-definition a list of non-isomorphic sink-definitions sS, also constrained 
to obey problem definition and minimality conditions (C1-C14), can be created with a second application of 
Leiterspiel. The pseudo-code for the resulting generation/enumeration is provided in Alg. 

As outlined above, in the first stage of the enumeration/generation algorithm, Leiterspiel is applied to grow 
subsets from ^ of size i incrementally in i until i = L. Some of the network conditions have the appropriate 
inheritance properties, and hence can be incorporated as constraints into the constraint function / in the Leiterspiel 
process. These include 

• no repeated edge definitions: If ^ C C ^ and "C has the property that no two of its edge definitions 

and {i\sS') have i — i', then so does SS. Hence, the constraint function / in the first application of 
Leiterspiel incorporates checks to ensure that no two edge definitions in the candidate subset define the same 
edge. 

• acyclicity: If ^ C C ^ and "C is associated with an acyclic hyper graph, then so is SS. Hence, the 
constraint function / in the first application of Leiterspiel checks to determine if the subset in question is 
acyclic. 

At the end of this first Leiterspiel process, some more canonical edge definition sets ^ can be ruled as non-minimal 
owing to (Cl), requiring that each source appears in the definition of at least one edge variable. 

For each member of the resulting narrowed list of canonical edge definition sets we must then build a list 
of canonical representative sink definitions W . This is done by first creating the (c^-dependent) set of valid sink 
definitions 

:= {(i, sS) 13 a directed path in ^ from % to at least one edge in (44) 


which are crafted to obey the minimality conditions (CIO) that the created sink (defined by its input which is the set 
of sources and edges in sS in the sink definition (i, sS)) must have at least one path in the hyper graph defined by 
^ to the source % it is demanding, and (C2) that is can not have a direct connection to the source it is demanding. 

Leiterspiel is then applied to determine canonical (lexicographically minimal) representatives of sink definition 
sets utilizing the associated stabilizer of the canonical edge definition set ^ being extended as the group, with 
the test function / handling the minimality conditions (C12)| and (C13) during the iterations. 


This second application of Leiterspiel to determine the list of canonical sink definition sets W for each canonical 
edge definition set ^ does not have a definite cap on the cardinality of each of the canonical sink definition sets W. 
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Input: number of sources K, number of non-source edges L 
Output: All non-isomorphic network instances 

Initialization: ^ = 0 ; 

Let ^ := ^ C {1,..., + L} \ {i} }; 

Let fi be the condition that J(i, ^), (i', ^') such that i = i'\ 

Let /2 be the condition of acyclicity; 

Let acting group G := S{i,...,x} x S{k+i,...,k+l}', 

Call Leiterspiel algorithm to incrementally get all candidate transversal up to L: 

Tl = Leiterspiel{G^ 
for each ^ e Tl do 
if ^ obeys (Cl) then 

Let := {(i, s^)\i G {1,..., AT}, C {1,..., AT + L} \ {i}, 3 a directed path in ^ from i to at 
least one edge in ^}; 

Let f[ be the condition (C12); 

Let /2 be the condition (C13); 

Let acting group G := x S{k+i,...,k+l}; 

Call Leiterspiel algorithm to incrementally get all candidate canonical representatives, i.e., transversals, 
up to no new element can be added obeying (C12,C13): Tk = Leiterspiel{G, 
for each W G Tk do 

if W) obeys (C3-C7) and (C14) then 

I ^ = 

end 

end 

end 

end 

Algorithm 1: Enumerate all non-isomorphic (K^L) networks using Leiterspiel algorithm. 


Rather, subsets of all sizes are determined incrementally until there is no longer any canonical subset that can obey 
the constraint function associated with (C12) and (C13). Each of the candidate canonical sink definition sets W 
(of all different cardinalities) are then tested together with ^ with the remaining conditions, which do not have the 
inheritance property necessary for incorporation as constraints earlier in the two stages of Leiterspiel processing. 

Any pair of canonical surviving each of these checks is then added to the list of canonical minimal 

non-isomorphic network coding problem instances. 

An additional pleasant side effect of the enumeration is that the stabilizer subgroups, i.e., the network symmetry 
groups |T9| , are directly provided by the second Leiterspiel. Harnessing these network symmetry groups provides 
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a powerful technique to reduce the complex process of calculating the rate region for a network coding problem 
instance | [T^ . 

Although this method directly generates the canonical representatives from the network coding problem equiv¬ 
alence classes without ever listing other isomorphs within these classes, one can also use the stabilizer subgroups 
provided by Leiterspiel to directly enumerate the sizes of these equivalence classes of (^, #^) pairs, as described 
above via the orbit stabilizer theorem. Experiments summarized in Table show that the number of isomorphic 
cases is substantially larger than the number of canonical representives/equivalence classes, and hence the extra 
effort to directly list only canonical networks is worthwhile. It is also worth noting that a node representation, 
utilizing a node based encoding of the hyper edges, would yield a substantially higher number of isomorphs. 


D. Modification to Other Problem Types 

A final point worth noting is that this algorithm is readily modified to handle listing canonical representatives of 
special network coding problem families contained within our general model, as described in ^II-A For instance, 
IDSC problems can be enumerated by simply defining ^ to have each edge access all of the sources and no other 
edges, then continuing with the subsequent sink enumeration process. It is also easily adapted to enumerate only 
directed edges and match the more restrictive constraints described in the original Yan, Yeung, and Zhang GD rate 
region paper. 


E. Enumeration Results for Networks with Different Sizes 

By using our enumeration tool with an implementation of the algorithms above, we obtained the list of canonical 
minimal network instances for different network coding problem sizes with N = K L <b. While the whole list 
is available we give the numbers of network problem instances in Tablewhere |^|, |^|, |^| represent 
the number of canonical network coding problems (i.e., the number of equivalence classes), the number of edge 
descriptions of network coding problems including symmetries/equivalences, and the number of node descriptions 
of network coding problems including the symmetries/equivalences, respectively. As we can see from the table, 
the number of possibilities in the node representation of the network coding problems explodes very quickly, with 
the more than 2 trillion labeled node network coding problems covered by the study only necessitating a list of 
consisting of roughly 750,000 equivalence classes of network coding problems. That said, it is also important to 
note that the number of non-isomorphic network instances increases exponentially fast as network size grows. For 
instance, the number of non-isomorphic general network instances grows from 333 to 485, 890 (roughly, an increase 
of about 1500 times), when the network size grows from (2,2) to (2,3). To provide an illustration of the variety 
of networks that are encountered. Fig. depicts all 46 of the 333 canonical minimal network coding problems of 
size (2, 2) obeying the extra constraint that no sink has direct access to a source. 

As a special class of hyperedge multi-source network coding problems, it is easier to enumerate IDSC networks, 
defined in Example in ^ Since we assume that all encoders in IDSC have access to all sources, we only need to 
consider the configurations at the decoders, which additionally are only afforded access to edges from intermediate 
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Fig. 8: All 46 non-isomorphic network instances of (2, 2) networks with the constraint that sinks do not have direct access to 


sources. 
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TABLE II: Number of network coding problems of different sizes: \^\ represents the number of non-isomorphic networks, \f^\ 
represents the number of isomorphic networks with edge isomorphism, and \ ^ | represents the number of isomorphic networks 
with node isomorphism. 


(K,L) 

1^1 

1^1 


(1,2) 

4 

7 

39 

(1,3) 

132 

749 

18 401 

(1,4) 

18027 

420948 

600 067 643 

(2,1) 

1 

1 

6 

(2,2) 

333 

1 270 

163 800 

(2,3) 

485 890 

5 787 074 

2 204 574 267 764 

(3,1) 

9 

31 

582 

(3,2) 

239 187 

2 829 932 

176 437 964 418 

(4,1) 

536 

10478 

12 149 472 

Total 

744 119 

9 050 490 

2 381 624 632 119 


TABLE III: List of numbers of IDS C configurations. |^| is the number of configurations in the node representation including 
isomorphisms, \f^'\ is the number of configurations in the edge representation including isomorphisms, and \^\ is the number 
of all non-isomorphic configurations. 


(K,L) 

2 

3 

\K\ 

l^'l 

1^1 

\K\ 

l^'l 

1^1 

2 

54 

12 

4 

4970 

234 

33 

3 

234 

24 

3 

443130 

4752 

179 


nodes. These extra constraints are easily incorporated into Algorithmic by removing the edge definitions, restricting 
to the unique one associated with the IDSC problems, and enumerating exclusively the sink definitions. 


We give the enumeration results for = 2,3 and L = 2,3 IDSC networks in Table III while the full list is 
available in | [39| . From the table we see that, even for this special type of network, the number of non-isomorphic 
instances grows very quickly. For instance, the number of non-isomorphic IDSC instances grows from 33 to 179 
(roughly, a factor of 6 increase), when the network size grows from (2,3) to (3,3). 


V. Rate Region Results for Small Networks 

With the list of minimal canonical network coding problems provided by the algorithm in the previous section 
in hand, the next step in our computational agenda was to determine each of their rate regions with computational 
tools. In this section, we describe a database we have created which contains the exact regions of all general 
networks with sizes N = K ^ L <b and all IDSC networks with sizes Ff = 2,3 and L = 2,3. 


A. Database of Rate Regions for all networks of size N = K -f L <h 

We begin by describing the experimental results we obtained by running our rate region computation software 
on all general hyperedge network instances of size N = K -f L <b. These problems consist of 744,119 canonical 
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TABLE IV: Sufficiency of codes for network instances: Columns 3-8 show the number of instances that the rate region inner 
bounds match with the Shannon outer bound. 


iK,L) 

1^1 

^s,2(A) 

^2^+1 (A) 

^2^+2 (A) 

^^+®(A) 

^2""+7A) 

^Unear 

(1,2) 

4 

4 

4 

4 

4 

4 

4 

(1,3) 

132 

122 

132 

132 

132 

132 

132 

(1,4) 

18027 

13386 

16930 

17697 

17928 

17928 

18027 

(2,1) 

1 

1 

1 

1 

1 

1 

1 

(2,2) 

333 

301 

319 

323 

323 

333 

333 

(2,3) 

485890 

341406 

403883 

432872 

434545 

- 

485890 

(3,1) 

9 

4 

4 

9 

9 

9 

9 

(3,2) 

239187 

118133 

168761 

202130 

211417 

- 

239187 

(4,1) 

536 

99 

230 

235 

476 

476 

536 

Total: 

744119 

473456 

590264 

653403 

664835 

- 

744119 


minimal networks, representing 9,050,490 networks in the edge W) encoding and 2 381 624 632 119 networks 
in the standard node representation, as indicated in Table [I^ For each non-isomorphic network instance, we calculated 
several bounds on its rate region: the Shannon outer bound the scalar binary representable matroid inner bound 
the vector binary representable matroid inner bounds and linear inner bound 

indicated in ^ if the outer bound on the rate region matches with an inner bound, we not only obtain the exact 
rate region, but also know the codes that suffice to achieve any point in it. The general code constructions from 
representable matroids follow a similar process in ng, P^ , where rate regions and achieving codes are investigated 
for MDCS. 

Though it is infeasible to list each of the 744,119 rate regions in this paper, a summary of results on the matches 


of various bounds is shown in Table |IV| The full list of rate region bounds can be obtained at | [38| and can be 
re-derived using | [40| . 

Several key observations kay be made from Table First of all the Shannon outer bound is proved to be tight 
for all networks of size N = K ^ L <b. Additionally, the results show that linear codes are sufficient to exhaust 
the entire capacity region for all of them, as indicated in column 2 and 8 in Table Furthermore, we investigate 
the number of networks whose rate regions are achievable by simple linear codes, e.g., binary codes (columns 3-7 
in Table [Iv]), and find that simple binary codes are capable of exhausting most of the capacity regions. 

For all (1,2) and (2,1) networks, scalar binary codes suffice. However, this is not true in general even when 
there are only one or two edge variables. For example, there are some instances in (3,1), (4,1), (2, 2) and (3, 2) 
networks for which scalar binary codes do not suffice. As we can see from Table |IV| as the vector binary inner 
bounds get tighter and tighter (i.e., as we move to the right in Table 113 , the exact rate region is established for 
more and more instances. That is, with tighter and tighter binary inner bounds, more and more instances are found 
for which binary codes suffice. 
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> H{Y2) 

> H{Y2) + H{Y^) 

> H{Y2) + H{Y^) 

> H{Yr) + H{Y2) + H{Y^) 

> H{Yr) + 2H{Y2)+2H{Yi) 

network instance A in ExampleUl 



Fig. 10: Comparison of rate regions ^*(A) (which equals to ^o(A)j and ^|(A) for the (3, 3) network instance A in Example 
when source entropies are HiY 2 )^ HiYf)) — (1,2,1) and the cone is capped by Ri -\- R 2 + Rz < 10; the white 

part is the portion that scalar binary codes cannot achieve. The ratio of over ^*(A) is about 99.57% for this choice 

of{H(Y,),H{Y2),H{Y^)). 


In order to provide a sample of the sorts of results available in the database | [3^ , the following example shows 
the various inner bounds on the rate region of a representative (3, 3) problem. 

Example 4: A 3-source 3-encoder hyperedge network instance A with block diagram and rate region <^*(A) 
shown in Fig.|^ 
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Fig. 11: Comparison of rate regions and for the (3,3) network instance A in Example^^when source entropies 

are H{Y 2 ),H{Y 3 )] = [1,1,2] and the cone is capped by R 1 +R 2 + R 3 < 10; the white part is the portion that scalar 

binary codes cannot achieve. The ratio of over ^J(A) is about 99.41%/or this choice of [H{Yi), H{Y 2 ), H{Y 3 )]. 


s,2 = ^*(A) n 


(45) 


First, scalar binary codes do not suffice for this network. The scalar binary coding rate region is 

i?i + i ?2 + -R 3 > HiXi) + 2H{Y2) + H{Y3) 

i?i + i?2 + i?3 > HiY^) + H{Y2) + 2H{Y3) 

One of the extreme rays in the Shannon outer bound on rate region is [Ri,R 2 ,Rz,H{Yi),H(Y 2 ),H{Y 3 )] = 
[1,1,1,0,0,2], This extreme ray cannot be achieved by scalar binary codes because no scalar code can 
encode a source with entropy of two into a variable with entropy of at most one. Fig.[i^ illustrates the 
gap between ,^*(A) and ^s, 2 {R) with a particular source entropy assignment. When source entropies are 
[H{Yi),H{Y 2 ),H{Y 3 )] = [1,2,1] and the cone is capped by i?i + i ?2 + i ?3 < 10, there is a clear gap between 
the two polytopes, though the inner bound occupies more than 99% of the exact rate region for this choice 


of [H{Y^),H{Y 2 ),H{Y 3 )]. 

Second, vector binary codes from 7 bits do not suffice for this network either. The vector binary coding 
rate region is 


n {i?i + i ?2 + i?3 > H{Y3) + 2H{Y2) + H{Y 3 )) . 


(46) 
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TABLE V: Sufficiency of codes for IDSC instances: Columns 3 and 4 show the number of instances that the rate region inner 
bounds match with the Shannon outer bound. 


{K,L) 

1^1 

^s.2(A) 

.«^+^(A) 

(2,2) 

4 

4 

4 

(2,3) 

33 

26 

33 

(3,2) 

3 

3 

3 

(3,3) 

179 

143 

179 


One of the extreme rays in the Shannon outer bound is [Rx,R- 2 ,,R:i,H{Yx),H(Y 2 ),H{Yz)] = [ 2 , 1 , 1 , 1 , 2 , 0 ]. 
This extreme ray cannot be achieved by binary codes from 7 bits because the empty source takes one bit 
as well when we assign bits to variables in general. Hence, at least 8 bits are necessary (2+1 +1 +1 +2+1 =8), 
as will be shown later. In our inner bound calculation, every variable in the network needs to have at least 
one associated element from the representable matroid, even though its entropy can be zero, like in 
this case. Though this inner bound is still loose in the sense of matching with the exact rate region, it is 
tighter than the scalar binary inner bound .^s, 2 (A). This is illustrated in Fig. [m by choosing a particular 
source entropy tuple. When source entropies are [H{Yi),H{Y 2 ),H{Y^)] = [ 1 , 1 , 2 ] and the cone is capped 
by + i ?2 + < 10, there is a clear gap between the two polytopes, though the scalar inner bound takes 

more than 99% space of the tighter vector binary inner bound for this choice of [H{Yi),H{Y 2 ),H{Y^)\. 

However, vector binary codes from 8 bits suffice for this network and thus One can 

construct vector binary codes to achieve all extreme rays in the Shannon outer bound on the rate region. 
For instance, the extreme ray [Ri,R 2 ,R 2 >,H{X),H{Y),H{Z)] = [ 2 , 1 , 1 , 1 , 2 , 0 ] can be achieved by the vector 
binary code as follows: Ui = [Yi + Y^^\ Y^^^],U 2 = Y^^"^ + Y^^\U 2 , = Y^^\ where are the two 

bits in source Y 2 - 


B. Database of Rate Regions for small IDSC instances 

Here, experimental results on thousands of IDSC (defined in Example in |II-A| ) instances are presented sepa¬ 
rately. We investigated rate regions for 219 non-isomorphic minimal IDSC instances representing 5130 isomorphic 
ones. These include the cases when (K^L) = (2, 2), (2, 3), (3, 2), (3, 3). Similarly, for the rate region of each non¬ 
isomorphic IDSC instance, we calculated its Shannon outer bound scalar binary inner bound ^s, 2 , and the 
vector binary inner bounds where N = K Y L = K \(o\. 

A summary of results on the number of instances for which the various bounds agree is shown in Table |V| The 
exact rate regions, their converses, and the codes that achieve them for all 219 non-isomorphic cases can be obtained 
at p9| and can be re-derived using | |40[ . For the non-isomorphic IDSC instances we considered, the Shannon outer 
bound is always tight on the rate regions, and the exact rate regions are obtained. Scalar binary codes also only 
suffice for the instances with L = 2 but not for all instances with L = 3. However, vector binary codes from binary 











36 


matroids on + 1 variables suffice for all the 219 instances. Thus, for the IDSC problems up to AT < 3, L < 3, 
vector binary codes suffice. 

After obtaining these massive databases of all rate regions for small networks, our next question is how to learn 
from them, and further, how to use them to solve more (larger) networks. For this purpose, we will develop in the 
following two sections notions of network hierarchy that enable us to relate networks of different sizes, their rate 
regions, and their properties with one another. 

VI. Network Embedding Operations 

In this section, we propose a series of embedding operations relating smaller networks to larger networks in a 
manner such that one can directly obtain the rate region of the smaller network from the rate region of the larger 
network. These operations will be selected in a manner that, due to this mapping, properties of the larger network 
can be considered to be inherited from small networks embedded within it. In particular, we will show that if a 
certain class of codes is insufficient to exhaust the rate region of a small network embedded in a larger one, then 
this class of codes will be insufficient to exhaust the rate region of the larger one as well. 


A. Definition of embedding operations 

The first operation is source deletion. When a source is deleted or removed, the source does not exist in the new 
network and the decoders that previously demanded it will no longer demand it after deletion. Fig. |12a| illustrates 
the deletion of a source. When source k is deleted, t will no longer require k. A particular example is shown in 
Fig. |13a| After deleting the source, the minimality conditions are checked to make sure the obtained network is 
minimal, and if not, an associated minimal network is found via a series of reductions according to Thm|^ 

Definition 6 (Source Deletion (A\k)): Fix network A = (^, S’, P). If source k e y \s deleted, then, the 

new network is minimal(A'), where A' = (^',^, with y' = y\k and \k,t e y). 


The next operation we consider is edge contraction. When an edge is contracted, the edge will be removed, and 
its head nodes are given direct access to all the inputs of the tail node. Fig. |12b| demonstrates the contraction of 
an edge. As it shows, when edge e is contracted, the head nodes it connects to will directly have access to all the 
input of its tail node. Minimality conditions need to be checked after this operation. A particular example is shown 
in Fig. |13b[ 

Definition 7 (Edge Contraction (A/e)): Fix network A = {y, W, y , S, 13). If edge e is contracted, then, the new 
network is minimal(A'), with A' = [y,^, y, S', [3) where S' = S\{e\J In(Tl(e))) Ug,^jj^^^^g^^{Tl(e'), Hd(e) U 
Hd(e')}. 


Finally, we define edge deletion. When an edge is deleted, it is simply removed from the graph, and the resulting 


graph is then checked and, if necessary, further reduced, for minimality. Fig. 12c demonstrates the deletion of an 
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(a) Source deletion: when source k is deleted, i 
previously required will no longer require it. 



(b) Edge contraction: when e is contracted, the head nodes directly 
have access to input of Tl(e). 


sends nothing to the network. Decoders that 



information from e. 


Fig. 12: Definitions of embedding operations on a network 


edge, and Fig. |13c| gives a particular example of the operation. With consideration of minimality conditions, we 
formally define the edge deletion. 

Definition 8 (Edge Deletion (A\e)): Fix network A = If edge e is deleted, then, the smaller 

network instance minimal(A'), with A' = {y, y, jd) where ^ \ e. 

Based on these operations, we make precise the notion of an embedded network, or a network minor. 

Definition 9 (Embedded Network): A network A' is said to be embedded in another network A, or is a minor 
of A, denoted as A' -< A, if A' can be obtained by a series of operations of source deletion, edge deletion/ 
contraction on A. Similarly, we say that A is an extension of A', denoted A >- A'. 

With this definition in hand, we set out in the next subsection on determining the relationship between the rate 
region and properties of a large network and the rate region and properties of a small network embedded within it. 

B. Inheritance of Rate Regions & their Properties Under Embedding Operations 

In this section we will prove a series of theorems that explain both how to obtain the rate region of an embedded 
network, under the operators defined in the previous subsection, from that of a larger extension network, as well 
as how certain properties of the rate region can be viewed as inherited under embedding operations. A particularly 
interesting rate region property we will consider is the sufficiency of a class of linear codes to exhaust the entire 
capacity region. Note that in each of the theorems below, the network A' will refer to the network in the definition 
of the associated operator (source deletion, edge contraction, and edge deletion) before the minimal(-) operator is 
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(a) Source deletion example: when si is deleted, its hyperedge is removed, and the sink t2 which previously Yi, Y2 will now demand only Y2. 
When minimality is considered, it will be observed that the new sink t2S ability to decode I2 has been implied by ti. Thus, t2 is removed as 
well. At this point U2 and U3 have become parallel edges, which are then merged. 



(b) Demonstration of edge contraction on a network: when 03 is contracted, the input of g2 will be directly available to ts. When minimality 
is considered, Y2 is now trivially decoded at ts due to direct access to it, and thus Y2 is removed from In addition, g2 is removed. 



(c) Demonstration of edge deletion on a network: when 02 is deleted, ^2, ts have no access to U2. Then fi, t2 are combined since they have the 
same input after deleting U2. 

Fig. 13: Examples to show the embedding operations on a network 


applied. 

Theorem 3: Suppose a network A" = minimal(A') is a minimal form of a network A' created by deleting 
source k from another network A = (^, , P), i.e., A' = A\k. Then for every I e {*, q, ( 5 , q), 0 } 

^i{A") = minimalA'^A" ,, ({R G ^z(A) \H{Yk) = 0})^ . (47) 

Proof: We will prove ^z(A') = ^ ({R G ^i{A) \H{Yk) = 0 }), since the remainder of the theorem 

holds from the minimality reductions in Thm. 

Select any point R' G ^*(A'). Then there exists a conic combination of some points in ^^{A') that are associated 
with entropic vectors in T^, such that R' = ^ where aj > 0,Vj. For each r' , there exist random 
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variables ^ where \i G ^ \ k , such that the entropy vector 

= H{^) ^ C |5G^\A:,eG^| 

is in rj^/, where N' = N — 1 is the number of variables in A'. Furthermore, their entropies satisfy all the 
constraints determined by A'. Define Yj^^^ to be the empty sources, HiYjf^) = 0. Then the entropies of random 
variables {Y^{^\ i G U Y^^^ will satisfy the constraints in A with H(Y^^^) = 0 and the entropy vector 

= H{sY) sY C |5 G e G I will be in FJ^ since adding an empty variable does not make 

an entropic vector to be non-entropic. Denote vj = [vp H{Y^^^) = 0], then Vj G <^*(A). Hence, by using the same 
conic combination, we have an associated rate point R = ^ ^ ^*(A) \H{Yk) =0}. Thus, we 

Vj (A) 

have ^^{N) C ^({R G ^^{l\)\H{Yk) = 0}). If R' is achievable by codes (scalar or vector), 

(i) 

there exists a construction of some basic codes (scalar or vector) to achieve it. Since letting Ys be empty does 
not affect the other sources and codes, the same construction of basic F^ codes will also achieve the point R with 
H{Y,) = 0. Thus, C e ^i{A)\H{Yk) = 0}),l G {q, {s,q)}. 

On the other hand, if we select any point R G {R G (A) |Ff (Y^) = 0}, then, there exists a conic combination of 

some points in {A)n{H{Yk) = 0} associated with entropic vectors in FJ^, i.e., R = ^ 

r,e^.(A)n{/f(yfc)=o} 

0, Vj. For each rj, there exist random variables such that their entropies satisfy all the constraints 

determined by A. Furthermore, since aj > 0, the only conic combination makes H{Yk) = 0 is the case that 
HiXk^) ~ because the entropies of satisfy all constraints determined 

by A' and the entropic vector projecting out Y^^^ is still entropic. Using the same conic combination, R' = 
Proj\^(y^) ajYj = ^ ^ Thus, we have Proj^\^(y^)^^({R G ^^{A)\H(Yk) = 

0 }) C If R is achievable by F^ code C, then the code to achieve R' could be the code C with deletion of 

rows associated with source Yk, i.e., C' = C\y^^.. Thus, ,^({R G ^i{A)\H{Yk) = 0}) ^ '^z(A'),/ G 

{g, (5,g)}. 

Furthermore, for any point R' G <^o(A'), there exists an associated point h' G and a rate vector r' = 
[Re\e G (o] such that R' = Pi'oj^\j:^(^y^) ^ [h',r'] fl ^a'- Clearly, if we increase the dimension of h' by adding 
a variable Y^ with zero entropy, i.e., HiY^) = 0. we have the new entropy vector in Fat. That is, if we define 
^ ~ ^^n{ys Ue\se^' ee<^}\^ — G e G , then h G Fat. Since H{Yk) = 0, the network constraints 

in A will be satisfied given that the zero entropy does not break the conditional entropies associated with network 
constraints. Hence, there exists an associated point R G <^o(A) with H{Yk) = 0. Therefore, we have <^o(A') C 
^ '^o(A)|Ff(yfc = 0)}). Reversely, suppose a point R G ^o{^) is picked with HiY^) = 0- 
There exists a vector h G Fat and a rate vector r = [Re\e G S’] such that R = Proj^ ^ [h, r] fl ^a- Since 
the network constraints ^a with H{Yk) = 0 will be ^as and Pi'oj^\j:^(^y^) ^ [h,r] G Fat' fl we have 

^ Therefore, we have Proj^^^^y^^ ,,({R G ^o(A)|Ff(yfc = 0)}) C ^^(A'). ■ 

Theorem 4: Suppose a network A" = minimal(A') is a minimal form of a network A' obtained by contracting 
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e from another network A = (^, , /3), i.e., A' = A/e. Then 

^i{k") = minimalA'^A" ^ G {*,g,o} (48) 

2 minimalA'^A" (Proj^ , (49) 


Proof: We will prove <^/(A') = Proj^ ({R G <^/(A)}) for / G {*, g, o}, and for the scalar case, ^s,q{^") ^ 
minimalA'^A" (^^^iuj,r\Re^s,q{^)^ ^ since the remainder of the theorem holds from the minimality reductions in 
Thm.E 

Select any point R' G ^*(A'). Then there exists a conic combination of some points in that are associated 

with entropic vectors in T^, such that R' = where aj > 0,Vj. For each r' , there exist random 

variables E S’ \e, such that the entropy vector 


hor 




^ C 




u 


U) 




I G 


^\e}] 


is in T%f, where A^' = — 1 is the number of variables in A'. Furthermore, their entropies satisfy all the 

constraints determined by A'. In the network A, define Ue^ to be the concatenation of all inputs to the tail node 


(i) _ ^ 


of e, = u. 


In(Tl(e)) 


. Then the entropies of random variables 




will satisfy the constraints in A, and 

er^. 


additionally obey Hence, £/ C |s e G 

That is, Tj = [r/, i?e > G .^*(A). By using the same conic combination, we have an associated rate 

point R = ^ ^ ^*{A) iRe > iT(u|^Lw Thus, we have 

V. .r-62> rA\ ^ K \ )) ) 


vj (A) 

^(AO c Proj„,,\^^({R G ^iA)\Re > ff(Ui„(Ti(e)))}) c Proj^,,\^^.^(A). 


(50) 


If R' is achievable by general codes, since concatenation of all input is a valid vector code, we have 


^,(A') c Proj„ ,,\^^({R G ^g(A)|i?e > -ff(Uin(Ti(e)))}) C Proj„ ,,\^^^,(A). (51) 

However, we cannot establish same relationship when scalar ¥q codes are considered, because for the point R', 
the associated R with H{Ue) may not be scalar F^ achievable. 

On the other hand, if we select any point R G {R G ^*(A)}, then, there exists a conic combination of 
some points in ^,^(A) associated with entropic vectors in F^, i.e., R = ^ ^ ^^^h 

Tj, there exist random variables such that their entropies satisfy all the constraints determined by 

A. Since the entropies of |y^\ |i ES\e^ satisfy all constraints determined by A' (because they are a 
subset of the constraints from A) and the entropic vector projecting out Ue is still entropic. Thus, by letting Re^ 
to be unconstrained, we have Proj^ ^^^^r^ G ^*(A'). Further, by using the same conic combination, R' = 
Pi*oj\i?e ^ ^ ^*(A'). Thus, we have Proj^ ({R G ^*(A)}) C ^^A'). 

If R G <^*(A) is achievable by F^ code C, either scalar or vector, then the code to achieve R' = Proj^ G 

S?^{A') could be the code C with deletion of columns associated with edge e, i.e., C' = C.^\Ue^ because the code 
on edge e is not of interest. Thus, we have Proj^ C ^^(A'), I G {g, (s,g)}. 
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Furthermore, for any point R' G ^o(A'), there exists an associated point h' G Fjv' and a rate vector r' = 
[Ri\i G (o \ e] such that R' = Proj^ [h^ r'] D ^a'- Clearly, if we increase the dimension of h' by adding 
a variable Ue which is the vector of all input variables to the tail node of e, i.e., Ue = [Ui\i G In(Tl(e))] and 
H{Ue) = H{Ui^i G In(Tl(e))), we have the new vector in F n. That is, if we define 


h = 


h' 






(52) 


[ ^'sYn{Ys,Ui\sey’^e<g'}U{Ui\ieln{T\{e))} Ue e ^ 

for C {Ys, Ui\s G i G S’}, then h G Fat. Further, we let Re to be unconstrained, i.e.. Re = oo. Since Ff(f/e) < 
Re, the network constraints in A will be satisfied given that the other constraints will not be affected. Hence, 
there exists an associated point R G ^o(A) with H{Ue) < Re^ where Re is unconstrainted. Therefore, we have 
'^o(A') C Proj^ G <^o(A)}). Reversely, suppose a point R G <^o(A) is picked with Re unconstrained. 

There exists an associated vector h G Fat and a rate vector r = G such that R = Proj^ ^ [h, r] D 
Since Re is unconstrained, we will have H{Ue) unconstrained as well. Since the network constraints with Re 
unconstrained will be ^as and Proj^ [h, r] G Pat' H ^as we have Proj^ ^\^^R G <^o(A'). Therefore, we 
have Proj^ ,,\^^({R G <^o(A)}) C <^o(A'). ■ 


Theorem 5: Suppose a network A" = minimal(A') is a minimal form of a network A' obtained by deleting e 
from another network A = {y, y, S, P), i.e., A' = A \ e. Then 

ypA') = minimaW^A" (Proj^^^\^^({R G ^z(A)|Re = 0})^ , I G {s,q),o} (53) 


Proof: We will prove ^/(A') = Proj^ ({R G ^/(A) \Re = 0 }), since the remainder of the theorem holds 
from the minimality reductions in Thm. 

Select any point R' G ^*(A'). Then there exists a conic combination of some points in that are associated 

with entropic vectors in F^, such that R' = where aj > 0,Vj. For each r' , there exist random 

r'.e^RA') 

variables G S\e, such that the entropy vector 


hO')' 


H{s2/) 


sY C 



ul^'’ |s G e ^\e}] 


is in F^,, where A^' = A^ — 1 is the number of variables in A'. Furthermore, their entropies satisfy all the constraints 


determined by A'. Let Ue^ be the empty set or encoding all input with the all-zero vector, Ue^^ = 0 and further 
let Re = 0. Then the entropies of random variables |y^\ will satisfy the constraints in A, and additionally 

obey H{Ue^^) < Re = Furthermore, the vector = |^i7(^) sY C \s G yp ^ ^ 

That is, Tj = [vpRe = 0] G ^*(A). By using the same conic combination, we have an associated rate point 
R = X] ajYj G {R G y^{A) \Re = 0}. Thus, we have ^*(A') C Proj^ ({R G ^*(A)|Re = 0}). 

If R' is achievable by general ¥q linear vector or scalar codes, there exists a construction of basic linear codes 
to achieve it. Since all-zero code is a valid vector and scalar linear code, we have ^z(A') C Proj^ ^\^^({R G 
'^/(A)|Re = 0}), ^ ^ {Q^{s,q)}- 


U) _ 
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On the other hand, if we select any point R G {R G ^*(A)|Re = 0}^ then, there exists a conic combination of 
some points in ^^{A) D {Re = 0} associated with entropic vectors in i.e., R = ^ 

rje^^iA)n{Re=0} 

0, Vj. For each r^, there exist random variables such that their entropies satisfy all the constraints 

determined by A. Furthermore, since aj > 0, the only conic combination makes Rg = 0 is the case that Re = 0 and 
further H{Ue^^) = 0. We can drop H{Ue^^), i.e.. Re, because the entropies of | satisfy all constraints 

determined by A' and the entropic vector projecting out Ue^ is still entropic. Using the same conic combination, 
R' = Proj\^^ 5]; ajYj = Proj^ R G ^*(A'). Thus, we have Proj^ G ^*(A)|Re = 0}) C 

vj (A) 

^*(A'). If R is achievable by ¥q code C, then the code to achieve R' could be the code C with deletion of columns 
associated with edge Ue, i.e., C' = Thus, Proj^ G ^i{A)\Re = 0}) C ^/(A'),/ G {q^ (s^q)}. 

Furthermore, for any point R' G ^o(A'), there exists an associated point h' G Tn' and a rate vector r' = 
[Ri\i G ^ \ e] such that R' = Proj^ [h^ r'] D ^a'- Clearly, if we increase the dimension of h' by adding 
a variable Ue with zero entropy, i.e., H{Ue) = 0, we have the new entropy vector in Fat. That is, if we define 
^ ~ ^'s^n{Ys Ui\sey' — {^s^'Ui\s G G S} , then h G Fat. Further, we let Re = 0. Since H{Ue) = 

Re = 0, the network constraints in A will be satisfied given that the zero entropy (capacity) does not break the 
conditional entropies associated with network constraints. Hence, there exists an associated point R G <^o(A) with 
H(Ue) = Re = 0- Therefore, we have <^o(A') C Proj^ G <^o(A)|Re = 0}). Reversely, suppose a point 

R G <^o(A) is picked with Rg = 0. There exists a vector h G Fat and a rate vector r = [Ri\i G (o] such that 
R = Proj^ ^ [h, r] fl ^a- Since Rg = 0, we will have H{Ue) = 0. Since the network constraints ^a with 
H{Ue) = Re = 0 will be ^as and Proj^ [h, r] G Fat' H^as we have Proj^ ^\^^R G <^o(A'). Therefore, 
we have Proj^^ G ^o{A)\Re = 0}) C ^o{A')- ■ 


Corollary 4: Consider two networks A, A', with rate regions ^*(A),,^*(A'), such that A' ^ A. If F, vector 
(scalar) linear codes suffice, or Shannon outer bound is tight for A, then same statements hold for A'. 
Equivalently, if F, vector (scalar) linear codes do not suffice, or Shannon outer bound is not tight for A', then 
same statements hold for A. Equivalently, if ,^i(A) = ^*(A), then ^i{A’) = ,^*(A'), for some I g {o,q, {s,q)}. 


Proof: From Definition we know that A' is obtained by a series of operations of source deletion, edge 
deletion, edge contraction. Theorems indicate that sufficiency of linear codes, vector or scalar, and the tightness 
of Shannon outer bound are preserved for each single embedding operation. For vector case, if ^%q{A) = .^*(A), 
(14§, Q directly give ^g(A') = ^*(A') for source deletion, edge contraction, and edge deletion, respectively. 
Similar arguments work for the tightness of the Shannon outer bound. For scalar code sufficiency, ( [47] ) and ( [5^ 
indicate the same preservation of sufficiency of scalar codes for source and edge deletion, respectively. For edge 
contraction and assumption of if = ^*(A), ( [48] ) and ( |49| ) indicate M^{A') C ^(A'). Together with the 

straightforward fact that ^ ^^{A'), since scalar codes achievable rate region must be subset of the 

entire rate region, we can see = <^*(A') holds for edge contraction as well. ■ 

Having introduced the embedding operations, which give smaller networks from larger networks, we next 
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(a) Sources merge: the merged source will 
serve for the new larger network. 



(b) Sinks merge: input and output of the 
sinks are unioned, respectively. 



(c) Intermediate nodes merge: input and output (d) Edges merge: one extra node and four associated 
of the nodes are unioned, respectively. edges are added to replace the two edges. 

Fig. 14: Combination operations on two smaller networks to form a larger network. Thickly lined nodes (edges) are merged. 





introduce some combination operations to get larger networks from smaller ones. 

VII. Network Combination Operations 

In this section we propose a series of combination operations relating smaller networks with larger networks in 
a manner such that the rate region of the larger network can be easily derived from those of the smaller ones. 
In addition, the sufficiency of a class of linear network codes is inherited in the larger network from the smaller 
one. Throughout the following, the network A = , ^5) is a combination of two disjoint networks 

Ai = A). ^ c {1,2}, meaning n ^2 = 0 , H ^2 = 0 , H ^ = 0 , A H <^2 = 0 , and 

A(^i) Cl A(^2) = 0, Vti G ^2 G 


A. Definition of Combination Operations 

The operations we will define will merge network elements, i.e., sources, intermediate nodes, sink nodes, edges. 


etc, and are depicted in Fig. Since each merge will combine one or several pairs of elements, with each pair 
containing one element from Ai and the other from A 2 , each merge definition will involve a bijection tt indicating 
which element from the appropriate set of A 2 is paired with its argument in Ai. 

We first consider the sources merge operation, in which the merged sources will function as identical sources for 


both sub-networks, as shown in Fig. 14a A sink requiring sources involved in the merge will require the merged 
source instead. 
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(a) Demonstration of source merge on two networks: sources si, S3 are merged 
to si, so Si will send information to both sub-networks. 



(b) Demonstration of sink merge on two networks: sinks ^2, ^4 are merged to 
t2, SO their input and demands are combined. 



(c) Demonstration of node merge on two networks: nodes g2 and are 
merged to node g2, so their input and output are also combined. 




(d) Demonstration of edge merge on two networks: when Di, t/4 are merged, 
one extra node and four edges are added to replace Ui,U4 in the two networks, 
respectively. 


Fig. 15: Example to demonstrate combinations of two networks. 
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Definition 10 (Source Merge (Ai.^ = A2.7r(^)) - Fig. |l4a| ): Merging the sources ^ c from network 
Ai with the sources 7r(^) c 5^2 from a disjoint network A 2 , will produce a network A with %) merged 
sources =5^ = u ^2 \ 7r(=5^), m) u ^ 2 , Mi) ^ = ^1 u ^ 2 , iv) = (A U <^2 \ ^) U where 

= {e G U (^2 |TI(e) G ^U7r(^)} includes the edges connected with the sources involved in the merge, 
^ = {(s,J^iUJ^ 2 )js G (s,c^i) G A, (7r(s),^2) ^ <^ 2 } includes the new edges connected with the merged 
sources, and v) updated sink demands 

/3i(t) t G 


I3{t) = 


(^(32it)\Tr{sfj Lin ^ (n(S) D (32(t)^ t e ^2 


Fig. 15a demonstrates the source merge in a network example. 


Similar to source merge, we can merge sink nodes of two networks, as demonstrated in Fig. 14b When two 
sinks are merged into one sink, we simply union their input and demands as the input and demands of the merged 
sink. 


Definition 11 (Sink Merge + A 2 .n{^)) - Fig. |l4b| ): Merging the sinks ^ C from network Ai 

with the sinks n{^) c ^2 from the disjoint network A 2 will produce a network A with i) ^ u ^ 2 ; 
^ u 5 ^ 2 , ii) -^ = n{^), in) S = SiU S 2 U \^, where , 2 / = {( 52 , u ^ 2)\92 & ^ 2 , c 

L ^,^2 c 3 ^2-,{g2-,n{^i) u ^ 2 ) e ^ 2 } updates the head nodes of edges in A 2 with new merged sinks, 
^ = {{92,^2) e ^21=^2 n n{^) 7^ 0 } includes the edges connected to sinks in n({^)), and v) updated sink 
demands 

Piit) t G \ ,i G {1, 2} 


/3(f) = 


/3i(f) U/?2(7r(f)) 


t G sr 


(54) 


Fig. 15b demonstrates the sink merge in a network example. 

Next, we define intermediate nodes merge. When two intermediate nodes are merged, we union their incoming 
and outgoing edges as the incoming and outgoing edges of the merged node, respectively, as illustrated in Fig.|14c| 


Definition 12 (Intermediate Node Merge (Ai.^ + A 2 . 7 r(^)) - Fig. |14c| ): Merging the intermediate node 5 G ^1 
from network Ai with the intermediate node n{g) g ^2 from the disjoint network A2 will produce a network 
A with i) y = y, ii) ^ = ^1 U ^2 \ n{g), Hi) ST = ,^1 U ,^2, w) <? = A U <^2 U .t/ U ^ \ \ 

where si = {{92,^2 \ T^{g) u g)\g2 e /^2, {92,^2 u n{g)) g S2} updates the head nodes of edges in A2 
that have n{g) as head node, SS = {(fi',=^ 2 )|( 7 r(fl),c^ 2 ) e ^2} updates the tail node of edges in A2 that 
have n{g) as tail node, = {e g (^2|TI(e) = n{g)} includes the edges in A2 that have n{g) as tail node, 
^ = {e G S’2\n{g) G Hd(e)} includes the edges in A2 that have n{g) as head node; and v) updated sink 
demands 


/3(f) = 


/32{t) t G ^2 


(55) 


Fig. 15c demonstrates the node merge in a network example. 
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Finally, we define edge merge. As demonstrated in Fig. |14d| when two edges are merged, one new node and 
four new edges will be added to create a "cross" component so that the transmission will be in the new component 
instead of the two edges being merged. 


Definition 13 (Edge Merge (Ai.e+A2.7r(e)) - Fig. |14d| ): Merging edge e G A from network Ai with edge 7r(e) G 
^2 from disjoint network A2 will produce a network A with i) ^ u ^2, ^ u ^2 u go, where go ^ 

^ 1,50 i '^ 2 , Hi) ^ = ^iU£^ 2 ,iv) S’ = (A\e)U(<? 2 \ 7 r(e))U{(TI(e),go),(TI( 7 r(e)),go),(flo,Hd(e)), ( 50 , Hd( 7 r(e)))}; 

and v) updated sink demands given by 1 


It is not difficult to see that this edge merge operation can be thought of as a special node merge operation. 
Suppose the edges being merged are Ai.e, A2.7r(e). If two virtual nodes ^ 1,^2 are added on e, 7r(e), respectively, 
splitting them each into two edges, so that e, 7r(e) go into and fiow out gi,g2, respectively, then, the merge of ^ 1 , ^2 


gives the same network as merging e,7r(e). Fig. 15d demonstrates the edge merge in a network example. 


B. Preservation Properties of Combination Operations 

Here we prove that the combination operations enable the rate regions of the small networks to be combined 
to produce the rate region of the resulting large network, and also preserve sufficiency of classes of codes and 
tightness of other bounds. 

Theorem 6: Suppose a network A is obtained by merging y with 7r(^), i.e., /Ki.y = A2.7r(^). Then 

^i(A) = Proj((^i(Ai) X ^((Aa)) n^o), ^ g, (s, g), o} (56) 

with ^0 = = H{Y„^g-)),ys e =5^|, and the dimensions kept in the projection are {H{Ys),s e y) and 

{Re, e e S), where S represent the source and edge sets of the merged network A, respectively. 


Remark 1: The inequality description of the polyhedral cone Proj((.^i x .^ 2 )n^o) for two polyhedral cones 
yj,j e {1,2} can be created by concatenating the inequality descriptions for and ^ 2 , then replacing 
the variable H{X^(^g-)) with the variable H{Xa) for each s & y. 


Proof: Select any point R e .^*(A). Then there exists a conic combination of some points in y„{k) that are 
associated with entropic vectors in such that R = ^ where aj > 0,Vj. For each rj, there exist 

random variables Y^j)€ S\e, such that the entropy vector 


h(i) 


H{sY) 


\sey,ieS^ 


is in F^, where N is the number of variables in A. Furthermore, their entropies satisfy all the constraints determined 
by A. When decomposing A into Ai,A 2 , let i.i.d. copies of variables Ys^\s G y work as sources 7 r(^) C ^2- 
The associated edges connecting y and nodes in ^2 will then connect 7 r(^) and nodes in ^ 2 - Then the random 
variables {Ys^^\ G G A}, {Yj^\ G ^2,^ ^ ^2} will satisfy the network constraints determined 
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by Ai, A2, and also ^o- Thus, R G Proj((^^(Ai) x ^*(A2)) Pi ^o)- Similarly, if R is achievable by codes, 
vector or scalar, the same code applied to the part of A that is Ai,A2 will achieve Ri,R2, respectively. Putting 
these together, we have ^z(A) C Proj((^/(Ai) x ^i{/\2)) G {*,g, 

Next, if we select two points Ri G R2 G <^*(A2) such that H{Ys) = G then there exist 

conic combinations Ri = V] . ^ for i = 1, 2, and for each j there exist a set of variables associated 

with sources and edges. Since H{Ys) = ^(T;r(s)) ^nd sources are independent and uniformly distributed, we can 
let the associated variables Yg and be the same variables. Then, after combination, the entropy vector 

of all variables {Ys^\ui^^\s G e G will be in F^. Furthermore, their entropies, together with the rate 
vectors from Ri,R2, will satisfy all network constraints of A, and there will be an associated point r = ri x r2 
with ^0- Using the same conic combination, we will find the associated point R = Ri x R2 D ^o- Hence, 
Proj((^*(Ai) X ^^{A2)) n^o) U ^^(A). Now suppose there exists a sequence of network codes for Ai and A2 
achieving Ri, R2. By using the same source bits as the source inputs for s in Ai and 77(5) in A2 for each s e S, we 
have the same effect as using these source bits as the inputs for s in the source merged A and achieving the associated 
rate vector R, implying R G ^z(A), I G {q^ (s, q)}, and hence ^s,q{A) P Proj((^s^g(Ai) x ^s,q{A2)) H^o) and 
^q{A) P Proj((^g(Ai) x^g(A2))n^o)- Together with the statements above, this proves ( [^ for I G {*, g, (s, q)}. 

Furthermore, by ( p 3 ] ), any point R G ^o(A), is the projection of some point [h, r] G Fjv H where h G Fat 
and r = [Re\e G Because the Shannon inequalities and network constraints in Fat H form a superset (i.e., 
include all of) of the network constraints in Fat H ^(A^), the subvectors [h\ Y] of [h, r] associated only with the 
variables in A^ (with being recognized as Yg for all s e S) are in Fat^ D ^(A^) and obey implying 
R G Proj((^o(Ai) X ^o(A 2)) n^o), and hence ^o(A) C Proj((^o(Ai) x ^o(A2)) H^o)- 

Next, if we select two points Ri G ^o(Ai), R2 G ^o{A2) such that H{Yg) = H{Y^(^s^^),\/s G then there 
exists [h*,U] G Fat^ n^(A^), where h* G Fat^ and U = [Re\e G ^i], such that R^ = Proj^. ^Jh*, U], i G { 1 , 2 } 
with = h\ (3) fo^ all s ^ S. Define h whose element associated with the subset sY of ./P = ^ U ^ is 
~ ^^n7r(y) U (oi, i e { 1 , 2 }. By virtue of its creation this way, this 

function is submodular and h G Fat. Since the two networks are disjoint, the list of equalities in ^3(A) is simply 
the concatenation of the lists in ^3(Ai) and ^3(A2), each of which involved inequalities in disjoint variables 
jyi and .yp, and the same thing holds for ^4/ with consideration of r\ Furthermore, since h* G ^2(Ai) and 
hy^ = G h obeys ^2(A). The definition of h, together with h* G ^i(A^), i G { 1 , 2 } and hy^ = 

hy , s G implies that h G ^i(A). Finally G ^(Ai) and G ^(A2) imply h G ^5 (A). Putting these 
facts together we observe that [h, r] G Fjvn=S?A> so R G .^o(A), implying .^o(A) ^ Proj((^o(Ai) x.^o(A2))n=S?o)- 
■ 

Theorem 7: Suppose a network A is obtained by merging sink nodes ^ with 7r(=#), i.e., (Ai.# + A2.7r{,#)). 
Then 

^i{k) = ^i{ki) X ^i{A 2 ), I e {*,q,{s,q),o} ( 57 ) 

with the index on the dimensions mapping from {e g S’ 2 \V\ 6 {e) g ttY)} to {e g (?|Hd(e) g ,#,TI(e) g 5 ^ 2 }- 
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Proof: Consider a point R G <^z(A) with conic combination of R = where aij > 0 for any j and 

I C {*, g, ( 5 , g), o}. Each vij has associated random variables or the associated codes. Due to the independence of 
sources in networks Ai, A 2 , and the fact that their sources and intermediate nodes are disjoint, the variables arriving at 
a merged sink node from Ai will be independent of the sources in A 2 and the variables arriving at a merged sink node 
from A 2 will be independent of the sources in Ai. In particular, Shannon type inequalities imply the Markov chains 

|Uin(t)nA5 Uin(t)n(f2) = ^(Y^jUin(t)nA) |Uin(t)nA 5 Uin(t)n^2) = |Uin(t)n^2) 

all t G ^ (even if the associated “entropies” are only in Eat and not necessarily f^). This then implies, together 
with the independence of the sources, that iJ(Y^(t)|Ui„(t)) = H{Y f}(^t)nj^i\^in{t)nSi) + 

showing that the constraints in ^ 5 (A) imply the constraints in and Furthermore, given the disjoint 

nature of A\ and A 2 , the constraints in ^i{A), are simply the concatenation of the constraints in ^i{Ai) and 
for i G {2, 3,4'}. Furthermore, the joint independence of all of , Y ^2 imply the marginal independence of the 
collections of variables Yand Y^ 2 ’ that ^i(A) implies ^i{Ai)^i G {1, 2}. This shows that Yi^j G x Y‘f - 
and further R G ^z(Ai) x Mi{^Af), and hence ^i{A) C ^i{Ai) x Mi{^Af),l G {*,g, (5,g),o}. 

Next, consider two points R^ G ^/(A^), i G {1, 2} for any I G {g, (5, g), o}. By definition these are projections 
of [h*,r*] e n^(Ai), [h*,r*] e r^. n^(Ai), [h*,r*] e Fat. n,5^(Ai), respectively, for i G {1,2}, 

where h* G FjVi and r* = [Re\e G (oi]. Define h with value associated with subset ^ C ^ of = 
+ ^^n^ 2 ’ 1^^^ ^1 easily verified that the resulting [h, r^,r^] G T^ ^ fl [h, r^,r^] G T^ fl 
[h, r^,r^] G Tat H respectively, (simply use the same codes from Ai and A2 on the corresponding parts 
of A). Since R = Proj^ ^[h, r^, r^], we have proven R G ^z(A), and hence that ^i{A) D ^z(Ai) x 
Further, for two points R^ G <^*(A^), i G {1,2}, there exist a conic combination of r}, R^ = ^ 

with associated random variables G ^ 1 , i G ^ ^ 2 , ^ ^ <^ 2 ^ satisfying the network 

constraints determined by Ai, A 2 . Due to the independence of sources and disjoint of edge variables, the union of 
variables in Ai,A 2 will satisfy the network constraints in the merged A. With the same conic combinations, we 
have R = ^ [ajrj,a|r|] G ^*(A). Thus, ^*(A) D ^*(Ai) x ^*(A 2 ). ■ 

Theorem 8: Suppose a network A is obtained by merging g and 7r{g), i.e., Ai.g + A2.7r(^). Then 

^i{A) = ^z(Ai) X ^/(A2), I G {*, g, (5, g), 0 } ( 58 ) 

with dimensions/ indices mapping from {e g ff 2 |Hd(e) = 7r{g)} to {e g ff|Hd(e) = g,TI(e) g ^ 2 } and from 

{e G ff 2 |TI(e) = 7r{g)} tO {e G |TI(e) = g, Hd(e) G ^2}. 

Proof: Consider a point R G ^i{A) for any I G {*,g, (5,g)} and all random variables associated with each 
component r/j in the conic combinations R = ^ where ai^j > 0 for any j and I G {*, g, (5, g), o}. 

Vj (A) 

The associated variables satisfy ^i(A),i = 1,3,4', 5. Partition the incoming edges of the merged node g in A, 
In(^), up into Ini(^) = In(^) fl Si the edges from Ai, and In 2 (^) = In(^) \ Ini(^), the new incoming edges 
resulting from the merge. Similarly, partition the outgoing edges Out(^) up into Outi(^) = Out(^) D Si and 



49 


Out 2 (^) = Out(^) \Outi(^). The ^3 constraints dictate that there exist functions /e such that for each e G Out(^), 
Ue = /e(t^ini(^), ^In2(^))- Define the new functions /' via 


/e(^Ini(^): ^In2(^)) — 


(59) 


/e(t^lni(^), 0 ) e G Outi(^) 

/e(0,f/in2(^)) e G Out2(5') 

i.e., set the possible value for the incoming edges from the other part of the network (possibly erroneously) to a 
particular constant value among their possible values - let’s label it 0. The network code using these new functions 
/' will utilize the same rates as before. The constraints and the topology of the merged network further dictated 
that were expressible as a function of i G {1,2}. In the remainder of the network (moving toward the 

sink nodes) after the merged nodes, at no other point is any information from the sources in the other part of the 
network encountered, and the decoders at the sink nodes in ^ need to work equally well decoding subsets of ^ 2 , 
regardless of the value of ^ 1 . Since the erroneous value for the f^ini(^) used for /', e G Out 2 (^) was still a valid 
possibility for some (possible other) value(s) of the sources in ^ 1 , the sinks must still produce the correct values 
for their subsets of ^ 2 - A parallel argument for ^ shows that they still correctly decode their sources, which were 


subsets of ^ 1 , even though the /gS were changed to /'s. Note further that (59) will still be scalar/vector linear if 
the original /gS were as well. 

However, since the /g/S no longer depend on the other half of the network, the resulting code can be used as 
separate codes for Ai and A 2 , given the associated rate points by keeping the elements in R associated with 
A^, i G {1,2} (or the associated rate points y\ - by keeping elements in Yi^j) in the natural way, implying that 
R G ^/(Ai) X ^i{/K 2 ). This then implies that ^/(A) C ^/(Ai) x ^/(A 2 ) for all I G (s,^')}. The opposite 

containment is obvious, since any rate points or codes for the two networks can be utilized in the trivial manner 
for the merged network. This proves ( [5^ for / G {*, ^, (<s, g)}. 

Next, consider any pair R^ G ^o(Ai) i G {1,2}, which are, by definition, projections of some [h% r*] G 
Vn- n^(A^), where h* G Vjsf. and r* = [Rg|e G S’i], i G {1,2}. Defining h whose element associated with the 
subset ^ C ^ is intersection respects the remapping of edges under the 

intermediate node merge, we observe that [h, r^,r^] G Ttv H and hence its projection R G <^o(A), proving 


^o(A) D ^g(Ai) X ^o(A2). 

Finally, consider a point R G ^o(A), which is a projection of some [h, r] G Ttv H where h G Tat and 
r = [Rg|e G (f]. For every ^ C define and define h' with h'^ = 

and R' = proj^ ^h'. We see that h* ^ G {1,2}, because conditioning reduces entropy and entropy is non¬ 

negative, but all of the conditional entropies at nodes other than g were already zero, while at g, the conditioning on 
the sources from the other network will enable the same conditional entropy of zero since the incoming edges from 
the other network were functions of them. This shows that R' G <^o(Ai) x ^o{/K 2 ). Owing to the independence 
of the sources proj^h = proj^h', while proj^h > proj^h' due to the fact that conditioning reduces entropy. The 
coordinate convex nature then implies that R G ^o(Ai) x showing that ^o(A) C ^o(Ai) x and 

completing the proof. ■ 
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Theorem 9: Suppose a network A is obtained by merging e and 7r(e), i.e., Ai.e + A2.7r(e). Then 

^z(A) = Proj\|g^^(g)|((^z(Ai) X ^^(A2)) n^o). ^ ^ (60) 

where = {^(Ti(i),^o) ^ > RjJ ^ {e,7r(e)}}, and projection dimension \{e,7r(e)} means 

projecting out dimensions associated with e,7r(e). Furthermore, ^(A^) x ^(A^) and viewed in the 

dimension of |^| + \j ^2 \ + 4 with assumption that all dimensions not shown are unconstrained. 


Proof: As observed after the definition of edge merge, one can think of edge merge as the concatenation of 
two operations: i) split e in Ai and 7r(e) in A 2 each up into two edges with a new intermediate node (g and 
7r{g), respectively) in between them, forming A'^ and A 2 , respectively, followed by ii) intermediate node merge of 
A'^.^ + A2.7r{g). It is clear that describes the operation that must happen to the rate region of A^, i G {1, 2} to 
get the rate region of A-, because the contents of the old edge e or 7r(e) must now be carried by both new edges 
after the introduction of the new intermediate node. Applying Thm. [^to A'^ and A 2 yields (60). ■ 

With Theorems [^ - [^ one can easily derive the following corollary regarding the preservation of sufficiency of 
linear network codes and tightness of Shannon outer bound. 


Corollary 5: Let network A be a combination of networks Ai, A 2 via one of the operations defined in If 
¥q vector (scalar) linear codes suffice or the Shannon outer bound is tight for both Ai, A 2 , then the same will 
be true for A. Equivalently, if e {1,2} for some I e {o, g, (s,g)} then also ^i{A) = ^*(A). 


Now we have defined both embedding and combination operators, and have demonstrated methods to obtain the 
rate regions of networks after applying the operators. Next, we would like to discuss how to use them to do network 
analysis and solve large networks. 


VIII. Results with Operators 

In this section, we demonstrate the use of the operators defined in and |VII| We will first discuss the use 
of network embedding operations (^V^ to obtain the forbidden network minors and to predict code sufficiency 
for a larger network given the sufficiency is known for the smaller embedded network. Then, we discuss the use 
of combination operations ( |VII| ) to solve large networks. Finally, we discuss how to use both combination and 
embedding operations together to obtain even more solvable networks. 

A. Use Embedding Operations to Obtain Network Forbidden Minors 

One natural use of embedding operations is to obtain rate regions for smaller embedded networks given the rate 
region for a larger network, as shown in since the rate regions of embedded networks are projections of the 
rate region of the larger network with some constraints. 

We can use the embedding operators in a reverse manner. From Corollary we observe that if a class of linear 
codes suffice for a larger network, then it will suffice for the smaller networks embedded in it as well. Equivalently, 
the insufficiency of a class of codes for a network is inherited by the larger networks that have this network 
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Fig. 16: Using embedding operations to predict the insufficiency of scalar binary codes for a large network: since the large 
network I and intermediate stage networks II, III contain the small network IV as a minor, the insufficiency of scalar 
binary codes for network IV, which is easier than network I to see, predicts the same property for networks III, II and I. 


embedded inside. This is similar to the forbidden minor property in matroid theory, which states that if a matroid 
is not ¥q representable, then neither are its extensions. Therefore, if we know a small network has the property 
that a class of linear codes does not suffice, then we can predict that all networks containing it as a network 
minor through some embedding operations, will have the same property, without any calculations. For instance, 
in Fig.[^ the small network IV is a (1,3) network for which scalar binary codes do not suffice, because when 
H{Yi) = 2,H{Ui) = H{Us) = H(U^) = 1, there is no scalar solution (but there is a vector solution). Since 
networks I, II, III contain it as a minor (through the operations in the figure), we know that scalar binary codes 
will not suffice for them, either. This is verified by our computations in ^V] 

As in matroid theory, we may have a collection of small networks that we know should be "forbidden" as a minor 
in larger networks, in the sense of ensuring the sufficiency of a class of linear codes. For instance, in |T^ , it was 
shown that for the thousands of MDCS networks for which scalar binary codes do not suffice, there are actually 
only 12 forbidden network minors. For general hyperedge MSNC problems, we also built similar relationships 
as shown in Fig.[^ As Table |IV] shows, the numbers of instances that scalar binary codes do not suffice for 
(1,3), (3,1), (2, 2), (3, 2), (2,3) networks are 5,10,32,121064,144484, respectively. Those insufficient instances 
should not be a minor of any larger network that scalar binary codes suffice, and hence are forbidden minors. To 


obtain a minimal list of network forbidden minors, we build a hierarchy among them. As Fig. 17 shows, there are 
37430 out of 121064 insufficient (3, 2) networks actually containing smaller forbidden network minors, of which 

14867 can be related by source or edge deletion and 22563 can be related by edge contraction. Similarly, there are 

37739 out of 144484 insufficient (2, 3) networks actually containing smaller forbidden network minors, of which 

22444 can be related by source or edge deletion and 37739 can be related by edge contraction. Those networks 
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( 3 , 2 ) 

239187 in total 


121064 118133 

insufficient sufficient 

/ \ 

37430 have 84864 have 
embedded no embedded 


( 2 . 3 ) 

485890 in total 

/\ 

144484 341406 

insufficient sufficient 

/ \ 

60183 have 84301 have 
embedded no embedded 


minors 


minors 


/ X 


minors 


14867 related 22563 related 
by source by edge 

or edge deletiori^ contraction 

I 

5 insufficient 32 insufficient 
( 3 , 1 ) ( 2 , 2 ) 




minors 


22444 related 
by source 
or edge deletion, 

32 insufficient 

(2,2) 


37739 related 
by edge 
contraction 

10 insufficient 

( 1 , 3 ) 


Fig. 17: Relations between networks of different sizes that scalar binary codes do not suffice. The deletion operation considers 
both source and edge deletion, while the contraction operation only considers edge contraction. 


that do not have smaller forbidden network minors are new ones. 


B. Use Combination Operations to Solve Large Networks 

As shown in |VII| the rate region of a combined network can be directly obtained from the rate regions of the 
networks involved in the combination. We show that the sufficiency of a class of linear codes are preserved after 
combination. Actually, if we know one of the networks involved in the combination has the property that a class 
of linear codes does not suffice, the combined network will have the same property, since that small network is 
embedded in the combined network and it can be obtained by deleting the other networks. 


Fig.[^ demonstrates the idea of solving large networks obtained by combination operations. It shows that the 
rate region of the large combined network can be obtained directly from the rate regions of smaller network. In 
addition, sufficiency of linear codes is preserved. 


Example 5: A (6,15) network instance A can be obtained by combining five smaller networks Ai 


of which the representations are shown in Fig. 18 The combination process is I) A 12 = 


A2.{ti/,t2'}; N) Ai 23 = Ai2.e4 + A3.67 with extra node and edges 64^,67/; III) A45 = A4.5'io + 

IV) A — Ai 23 .{Xi,..., — A 45 .{Ai/,..., Ag/}. From the software calculations and analysis |40| , ED’ 

one obtains the rate regions below the 5 small networks. According to the theorems in ^VM] the rate region 
^*(A) for A obtained from ^*(Ai),.. is depicted next to it. Additionally, since calculations showed 

binary codes and the Shannon outer bound suffice for A*, i e {1,..., 5}, Corollary [^dictates the same for 
network A. 
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R. 

> 

H{X 2 ),i = l,2 

Rl -h R 2 

> 

H{Xi) + H{X 2 ) 

Rr 

> 

H{X4),i = 3,i,4' 

R^ + R. 

> 

H(X 3 ) + H{X 4 ),i = i,i' 

R:i + Ri + i?5 

> 

H{X3) + 2H(Xi),i = 4,4' 

Rg 

> 

H{Xe) 

Ri+Rn 

> 

H{Xe),i = 7,r 

Rg + Ri 

> 

H(X 3 ) + H(X 3 ],i = 7,r 

Rs + Ri 2Rg 

> 

H{X 3 ) + 2H(Xe)J = 7J' 

R 9 

> 

HiX3) 

Rio 

> 


Ru 

> 

HiXs) 

Ri 2 

> 

H{X4)+H(X3) 

Rio> + Ru + Ri 2 

> 

H(X 4 ) + 2H(X3) + H(Xe) 


{ 2 , 3 } { 1 , 2 , 4 } { 4 } { 6 } { 5 } { 6 } 



R 


*R2 


f9 


{1.2} {2} 
Ri > H{X2) 





m 

{ 4 } { 4 } { 3 } 

Ri >H{Xi),i = -iA 
R:i + R^>H{XA + H{X4) 

R2>H{X 2 ) 2i?,, > H{Xi) + 2H{X4) R^ + R7> HiX^) + //(Xg) 

Ri+R 2> HiXi) + H{X2) R^ + R7 + 2/?8 > H{XA + 2H{Xe) 


{ 6 } { 6 } { 5 } 
Re > H{Xe) 



{2',3’} {r,3'} 

i?9 > //(X 3 .) 


^12 

( 1 ^ 

( 5 ’} { 6 '} { 4 ’ 5 ’} 


R, + R, > H(X,) + H(X,.) + H(X,.) «n ^ ^ ^ 

Hi 2 > H{X4>) + H{As') 

Rio' + Ru + Ri2 > H{X4’) + 2H{X5') + H{Xe>) 


Fig. 18: A large network and its rate region created with the operations in this paper from the 5 networks below it. 


C. Use combination and embedding operations to generate solvable networks 

The combination operators provide a method for building large networks directly from smaller networks in such 
a way that the rate region of the large network can be directly obtained from those of the small networks. The 
embedding operators provide a method for obtaining a small network from a large network in such a way that the 
rate region of the small network can be expressed in terms of the rate region of the large network. These facts 
indicate that we can obtain networks by integrating combination and embedding operations. If we start with a list 
of solved networks, all networks obtained in the following combination and/or embedding process will be solvable. 
Note that one can apply these operations, especially the combination operations, an infinite number of times to obtain 
an infinite number of networks. For demonstration purposes, we would like to limit the size of networks involved 
in the process. As |VII| shows, it is not difficult to predict the worst case network size after combination. Since the 
combined network may have redundancies, the worst case here means there is no redundancy after combination 
so that the network size is easy to predict. For instance, after merging k sources of a (iT, L) with k sources of a 
K', L' network, the network size after merging will be K K' — k L A- L' in the worst case. We define the 
worst case partial closure of networks as the networks obtained in the combination and embedding process such 
that no network involved exceeds the size limit. An algorithm to generate the worst case partial closure of networks 
is shown in Algorithm As it shows, as long as the predicted network does not exceed the preset size limit, it 
will be counted as a new network and will be used as a seed, as long as it is not isomorphic to the existing ones. 







54 



(there ore 6) 




^ = ^-4 


All (2,2) minimal network coding problems that 
ore direct combinations of (1,1 ),(2,1 ),and (1,2) problems. 
(There ore only 3 out of 333 total (2,2) problems) 


Fig. 19: There are a total of 3 minimal (2, 2) network coding problems directly resulting from combinations of the 6 small 


network coding problems with sizes (1,1), (1,2), and (2,1). However, as shown in Fig. 20 by utilizing both combinations 
and embeddings operators, far more (2, 2) cases can be reached by iteratively combining and embedding the pool of networks 
starting from these 6 (1,1), (1, 2), and (2,1) networks via Algorithm^^ 


We start with a seed list of networks (seedList = prevAdd), then through combinations and embeddings of these 
networks, a list (curAdd) of new networks (newNef) will be generated, which will, in turn, be used as seeds 
again. The list of networks (curLisf) will be updated after each iteration. The process stops when there is no new 
network that would not exceed the size limitations (size cap) found, in the sense of closure under these operations. 
This tool is able to generate a large number of network from even small seed lists. For instance, if we use as a seed 
list the single (1,1) and single (2,1), together with the four (1,2) networks, and set the size limit for intermediate 
networks io K < 4, L < 4, there will be 11635 new networks generated. Even when embedding operations are not 
allowed, there will be 568 new networks. The details on the number of networks generated for different size limits 
is shown in Table 


From Table VI we first see that these six networks can generate a very large number of larger networks (see 


the bottom row of the table). We also see that the number of networks generated grows, even for small target 
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Fig. 20: The path of operations on a seed list of small networks to get a (2,2) network that cannot be directly obtained by 
simple combination. The size limits on networks involved in the operation process is K < 3, L < 4. 
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Input: Seed list of networks seedList, size limits on number of sources and edges 

Output: All network instances generated by combination and/or embedding operations on the seed list 

Initialization: network list for previous round prevList = 0, new networks from previous round 
prevAdd = seedList, current list of networks curList = 0, new networks generated in current round 
cur Add = 0; 

while size{prevAdd) > 0 do 

for every pair ^ x ^ G prevAdd x prevAdd U prevAdd x curList do 
if prediction of network size after merge does not exceed size limits then 
consider source, sink, node, edge merge on ^^ 
convert the new network to its canonical form newNet ; 
if newNet ^ curList then 

cur Add = cur Add U newNet', 

end 

end 

end 

for ^v^ry G prevAdd do 

consider source deletion, edge deletion and edge contraction on 
convert the new network to its canonical form newNet ; 
if newNet ^ curList then 

cur Add = cur Add U newNet', 

end 

end 

prevAdd = cur Add', 
prevList = curList', 
curList = curList U cur Add', 

end 

Algorithm 2: Generate all networks from a seed list of small networks using combination and embedding 
operations. 


network sizes, rapidly as the cap on the largest network size is increased. Furthermore, it is important to note that, 
when trying to calculate the rate regions of larger networks from a list of rate regions for smaller networks, both 
combinations and embeddings are useful. As Table |Vj shows, when no embedding operations are allowed in the 
generation process, the number of reachable networks are much less than those with embedding operations. To 
further see a demonstration of this fact, consider all 6 canonical minimal network coding problems of dimensions 


(1,1), (1,2), (2,1) as depicted in Fig. 19 There are only 3 out of 333 networks with size K = 2,L = 2 can 
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TABLE VI: The number of new canonical minimal network coding problems that can be generated from the 6 smallest canonical 
minimal network coding problems (the single (1,1) network, the single (2,1) network, and the four (1, 2) networks), by using 
combination operators (left), and both combination and embedding operators (right), in a partial closure operation where the 
largest network involved in a chain of operations never exceeds the “cap’' (different columns). 



combination operators only 

embedding and combinations 

size\cap 

(3,3) 

(3,4) 

(4,4) 

(3,3) 

(3,4) 

(4,4) 

(1,3) 

4 

4 

4 

4 

4 

4 

(1,4) 

0 

10 

10 

0 

10 

10 

(2,2) 

3 

3 

3 

8 

15 

16 

(2,3) 

13 

16 

16 

30 

131 

155 

(2,4) 

0 

97 

101 

0 

516 

648 

(3,2) 

2 

3 

2 

4 

10 

11 

(3,3) 

24 

24 

24 

42 

353 

833 

(3,4) 

0 

135 

135 

0 

2361 

5481 

(4,2) 

0 

0 

3 

0 

0 

3 

(4,3) 

0 

0 

17 

0 

0 

44 

(4,4) 

0 

0 

253 

0 

0 

4430 

all 

46 

292 

568 

88 

3400 

11635 


be reached by combination of smaller networks using only combination operations. The three networks and their 


smaller components are shown in Fig. 19 However, if we are allowed to use both combination and embedding 
operation together, we will reach more networks that are not reachable by merely combinations. As Fig. [20| shows. 


though we still use the same network pool as in Fig. 19 another (2,2) network is obtained by first combining 
smaller networks to a larger size and then using embedding operations to decrease the network size. Several steps 


of combination and embedding operations are necessary to get this network, and Fig. 19 shows the path through 
the operations from the initial seed list and the intermediately obtained network to reach it. As Table |Vj shows, 
there are at least 12 (2,2) networks which can be obtained in this manner. 


IX. Conclusions and Future Work 

This paper investigated the enumeration, rate region computation, and hierarchy of general multi-source multi¬ 
sink hyperedge networks. The network model includes several special ones such as independent distributed storage 
systems and index coding problems. This definition is further refined to a notion of minimal networks, containing 
no redundant sources, edges, or nodes, whose presence directly determines the rate regions from minimal networks. 
Furthermore, since networks related to one another through a permutation of the edge labels are derivable from 
one another, a notion of network equivalence or isomorphism under group action is defined. By harnessing the 
Leiterspiel algorithm, which calculates the orbits of subsets incrementally in subset size, an efficient enumeration 
algorithm is presented for enumerating non-isomorphic networks, i.e., canonical representatives of the equivalence 
classes under this isomorphism, directly. Using this algorithm, millions of non-isomorphic networks are obtained 
that represent trillions of network coding problems. Then by applying computation tools, exact rate regions of 
most of them are obtained, leaving the rest with outer bound and simple code achieving inner bounds. Only binary 
codes are considered here, and binary codes are shown to suffice for most of the networks under consideration. In 
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order to better understand and analyze the huge repository of rate regions, a notion of network hierarchy through 
embedding and combination operators is created. These operations are defined in a manner such that the rate region 
of the network after each operation can be derived from the rate region of each of the networks involved in the 
operation. The embedding operations enable us to obtain a list of forbidden network minors for the sufficiency of 
a class of linear codes. It is shown that for many networks that scalar binary codes do not suffice, they contain 
a smaller network, for which scalar binary codes also do not suffice, as a minor under the embedding operations. 
The combination operations enable us to solve large networks that can be obtained by combining some solvable 
networks. The integration of both embedding and combination operations is able to generate rate regions for even 
more networks than can be solved directly with combination alone. These operations open a door to many new 
avenues of network coding research. Some of the pressing future problems for investigation include: I) assessing the 
coverage of the operators in the space of all problems; II) if necessary, the creation of more powerful combination 
operations, such as node and edge merge, source and sink merge, etc; III) a notion of forbidden minors which can 
harness both combination and embedding operators. 
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Appendix A 

Complete proof of Theorem [T] 
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A. Converse 

We need to prove that for any achievable rate tuple R G we have R G Projr^^(con(rj^ D ^13)0^40^5). 

Pick a point R = Ri, ..., Rl] that is in ^c(A). For convenience in comparing with the 

notations in (m, we let ujs = H{Ys) be the source rate to achieve. Let an arbitrarily small e > 0 be given. Since 
R is achievable, for all sufficiently large n, there exists a block n code such that 

+ e€4, (61) 

n 

^ ^ s G ^ (62) 

< e, (63) 


where r]e is the index set of messages sent on edge e and Tq is the transmitted source rate at source s under this block 
code. For all s G we let Ys^^ be the block source variable which takes value from the set {1,..., }. For all 

e G (ft/, we let be the codeword sent on edge e and denote the alphabet of as = {0? • • •: — !}• 

From ( [^ , we know for all e G (ft/, 

< log I'^el 
= log(7/e) 

<n{Re^e). (64) 


For all source 5 G from ( [62| ) we have 

nuj, > = log > n{LO, - e). 

Since it is assumed that R is achievable, there exist random variables , C/i”^ such that 


(65) 


HiY^p) = (66) 

m^”^|ui:;Ti(e))) = 0,ee^u (67) 

Following Lemma 1 in m (note that the input of a sink t may include some other sources that not in 
which applies the Fano’s inequality, we have 

( 68 ) 

where (j)t{n^e) has the following properties: 

1) (j)t{in^e) is bounded; 

2) 0t(n, e) —> 0 as n ^ oo and e ^ 0; 

3) 0t(n, e) is monotonically decreasing with increase of n and decrease of e. 
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From ( [64| ) - ( [68] ) we get the existence of entropic vector such that 


h-Y^ = hy^ 

(69) 



^t/e|Ui„(Tl(e)) = 0,ee4/ 

(70) 

hvs ^ n{ujs — e), 5 G ^ 

(71) 

^Y^wlUinw < n(Pt{n,€),'ite ^ 

(72) 

hjje — n{Ree)^ e E S’ 

(73) 

Now define the following two regions in that depend on n, and view the [Re\e G Su] 

as fixed vector: 

^ 4 ^^, = {h e ; hu^ < n{Re + e), e e ,^b}, 

(74) 

^ 5 ", = {h e < nMn,e),yt € 

(75) 

Then from (|69|)- (|7^ we see that there exists 


her^ 

(76) 

such that 


hG^i3n^4^,n^5^, 

(77) 

and \ls ^ y 


hvs ^ Ti{uJs — e), 

(78) 

i.e. 


hy 

-^>uJs-e. 

(79) 

n 


From ([76]) and ([77]) we obtain 


hGr^n^i3n^4^,n^5^,. 

(80) 

Since F^ D ^13 contains the origin, we get that 


n“^h e con(r;|;^ n ^ 13 ) n ^ 4 ,e n ^ 5 ,e, 

(81) 

where 


°^ 4 ,e = {h G ^ : hjj^ < + ^} 

(82) 

and 


= {h G : ^/3(t)|Uin(t) - 4>t{n,e)R G ^}. 

(83) 

For all n and e, define the set 


^(n,e) _ ^ con(r^ n SSis) n n — e, Vs g ^}. 

(84) 


The fact that is closed and bounded follows immediately from a similar proof in | [ll| . Then, we conclude 

that is compact. 
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Now from the fact that 0t(n, e) is monotonically decreasing in both n and e, so for all e' < e and n, 

and 

( 86 ) 




Note that from (78) and (^) we see that for any e > 0, is nonempty. Since is compact and 

nonempty, 

0 oo 

lim lim =00 

e^O n^oo I I I I 

is also nonempty and compact, which equals to 


e n=l 


(87) 


{h G con(rj^ n ^13) n ^4 n ^5 : hy^ = Co’s, Vs ^ ^}, 

where ^4 = {h G R2~-l • ^c/e ^ Re} as defined in | [ll| with [Re\e G c^u] as constants. 
Hence, if we let the i?e, e G (ft/ be unconstrained variables, we conclude that 


( 88 ) 


R G Proj^ ,^(con(r^ (0 ^ 13 ) H ^ 4 ' H ^ 5 ), 
where r = (Re, e G S’u) and u = (i7(Fs), s G ^). 

R. Achievability 


(89) 


We need to prove that for any point R G Proj^ ^(con(rj^ (0 ^ 13 ) (0 ^ 4 / (0 ^ 5 ), there exists a code such that 
this rate is achievable. 

Similar as Lemma 3 in m if we define 


= con(r;|;^ n ^13) 


and 


(90) 


(91) 


^2 = R(r^n^i3), 

where D{s^') = {ah : h G sz/' & 0 < a < 1}, we have 

M = ^ 2 - (92) 

The proof of ( [92| ) is identical to the proof of Lemma 3 in m except that we only have ^13 but [ [TT| | has ^ 123 . 


Let R be the point picked, we also have R G Proj^ ^(R(r^ (0 ^ 13 ) (0 ^ 4 / (0 ^ 5 ), then there exists an h G 
R(rj^ n ^ 13 ) n ^ 4 / n ^5 such that R = Proj^ ^(h). Furthermore, there exists an entropic vector hGr^n^ia 
and an a such that h = ah. 

Since h G F^ D ^13, there exists a collection of random variables ^ = {Ys,Ue\s G ^,e G (ou} such that 

ahy^ =Co’s, 5 G ^ (93) 

hY^ = ^hy. (94) 

hc/e|Ui„(Tl(e))=0’e G 


(95) 
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where ujg = H{Ys) is the source rate to achieve at source s. 

Furthermore, since h G ^4 fl ^5 and we only need to show h is asymptotically achievable, we have 

^ Re “h C G (96) 

ahY3(t)|Ui„(t) (97) 

where /i and 7 are arbitrarily small positive numbers. 

Our next step is to show that the rate vector R' = [cc^, Re + /^|<s G e G (f] is achievable. Then as 7 , /i ^ 0, 
we know that R is achievable. 

Use the similar code construction and performance analysis as in the proof of achievability in pT| , we can show 
that [ujs\s G is achievable if we set [Re + /i|e G as the capacities on edges. Encoding functions at sources 
are simply identity functions. Equivalently, the similar construction shows that R' is achievable. Therefore, R is 
achievable as 7 , /i ^ 0 . 


